update the codebase poc ver1
This commit is contained in:
24
.gitignore
vendored
24
.gitignore
vendored
@@ -212,3 +212,27 @@ secrets/
|
|||||||
*.claude
|
*.claude
|
||||||
*.kilo
|
*.kilo
|
||||||
*.zip
|
*.zip
|
||||||
|
tmp/
|
||||||
|
corpus/
|
||||||
|
corpus_db/
|
||||||
|
.locks/
|
||||||
|
models--gliner-community--gliner_small-v2.5/
|
||||||
|
models--microsoft--deberta-v3-small/
|
||||||
|
ner_model/
|
||||||
|
gemma_emb/
|
||||||
|
.cursor/
|
||||||
|
.husky/
|
||||||
|
Policy_Analysis/
|
||||||
|
.vscode/
|
||||||
|
package-lock.json
|
||||||
|
package.json
|
||||||
|
.env.development
|
||||||
|
dist/
|
||||||
|
run.sh
|
||||||
|
workspace/sprint_1_2/CODEBASE/knowledge/implementation/model/
|
||||||
|
*.onnx_data
|
||||||
|
*.onnx
|
||||||
|
*.bin
|
||||||
|
*.safetensors
|
||||||
|
*.pt
|
||||||
|
*.pth
|
||||||
@@ -17,3 +17,4 @@ Create up to 5 shareable agent skills under `PILOT_PROJECT/AGENT_SKILL/` that au
|
|||||||
| interface_contract_hygiene | Contract files, versioning, breaking-change policy |
|
| interface_contract_hygiene | Contract files, versioning, breaking-change policy |
|
||||||
| secrets_and_phi_safety | Secrets loading, PHI exclusion, CI safety, gitignore |
|
| secrets_and_phi_safety | Secrets loading, PHI exclusion, CI safety, gitignore |
|
||||||
| design_pattern_compliance | Layered arch, DI, Zustand, Terraform IaC, pinning |
|
| design_pattern_compliance | Layered arch, DI, Zustand, Terraform IaC, pinning |
|
||||||
|
| agent_tool_execution | Exa/Supabase/MedGemma agent tools, BFF proxy, MediaPipe output grammar |
|
||||||
53
AGENT_SKILL/agent_tool_execution/SKILL.md
Normal file
53
AGENT_SKILL/agent_tool_execution/SKILL.md
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
# Skill: agent_tool_execution
|
||||||
|
|
||||||
|
## Trigger Conditions
|
||||||
|
|
||||||
|
Any change involving Gemma-4-E2B agent tools, Exa search, Supabase RAG RPC, or MedGemma escalation.
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
### 1. Tool catalog is authoritative
|
||||||
|
|
||||||
|
- Only use: `exa_search`, `supabase_query`, `escalate_medgemma`.
|
||||||
|
- Never invent tool names in prompts or handlers.
|
||||||
|
- Bump `TOOL_CATALOG_VERSION` on breaking schema changes.
|
||||||
|
|
||||||
|
### 2. Secrets and egress
|
||||||
|
|
||||||
|
- `EXA_API_KEY`, `SUPABASE_SERVICE_ROLE_KEY`, MedGemma keys: **backend only**.
|
||||||
|
- Browser calls BFF routes; never `https://api.exa.ai` directly from the client.
|
||||||
|
- PHI-scrub queries before any network egress ([`secrets_and_phi_safety`](../secrets_and_phi_safety/SKILL.md)).
|
||||||
|
|
||||||
|
### 3. Exa defaults (Pattern 1 — raw retrieval)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "auto",
|
||||||
|
"numResults": 10,
|
||||||
|
"contents": { "highlights": true }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- Canonical reference: https://docs.exa.ai/reference/search-api-guide-for-coding-agents
|
||||||
|
- Do not use deprecated params (`useAutoprompt`, top-level `highlights`, `livecrawl: "always"`).
|
||||||
|
- `outputSchema` synthesis is BFF-only enrichment; not the default agent-loop path.
|
||||||
|
|
||||||
|
### 4. Retrieval policy (NFR-18)
|
||||||
|
|
||||||
|
- `supabase_query` first for in-corpus MOH guideline grounding.
|
||||||
|
- `exa_search` for external/recency queries.
|
||||||
|
- Final clinical answers must cite chunk IDs (Supabase) or URLs (Exa).
|
||||||
|
|
||||||
|
### 5. MedGemma escalation
|
||||||
|
|
||||||
|
- Requires consent + audit (NFR-16a) before `escalate_medgemma`.
|
||||||
|
- No direct Modal URLs from the browser.
|
||||||
|
|
||||||
|
### 6. MediaPipe agent output
|
||||||
|
|
||||||
|
- Parse tool calls from the **answer channel** when CoT is enabled.
|
||||||
|
- One block per turn: `<tool_call>` OR `<final>`.
|
||||||
|
|
||||||
|
## Package location
|
||||||
|
|
||||||
|
[`ml/implementation/nlp/agent_runtime/`](../ml/implementation/nlp/agent_runtime/)
|
||||||
@@ -17,25 +17,30 @@ Build a reproducible, air-gapped-first musculoskeletal ultrasound analysis platf
|
|||||||
### Functional (Sprint 1-2)
|
### Functional (Sprint 1-2)
|
||||||
- [ ] FR-25: Load knee DICOM → segment joint structures → measure synovium thickness → grade synovitis 0-3
|
- [ ] FR-25: Load knee DICOM → segment joint structures → measure synovium thickness → grade synovitis 0-3
|
||||||
- [ ] Grad-CAM overlay on primary viewport (zero extra clicks)
|
- [ ] Grad-CAM overlay on primary viewport (zero extra clicks)
|
||||||
- [ ] Circuit-breaker Socratic dialogue (radiologist challenges AI grade before finalizing)
|
- [ ] Circuit-breaker Socratic dialogue (radiologist challenges AI grade before finalizing) across 3-tier LLM system (Edge Gemma → Gemini → MedGemma)
|
||||||
- [ ] BERT drift monitor against baseline MOH corpus
|
- [ ] BERT drift monitor against baseline MOH corpus
|
||||||
- [ ] RAG-Referee validates every LLM-generated explanation against top-k retrieved MOH guideline chunks
|
- [ ] RAG-Referee validates every LLM-generated explanation against top-k retrieved MOH guideline chunks (mandatory for cloud tiers, lightweight for edge)
|
||||||
- [ ] Decree 13 PII scrubbing on all outbound text (client-side + FastAPI middleware)
|
- [ ] Decree 13 PII scrubbing on all outbound text (client-side + FastAPI middleware)
|
||||||
- [ ] ladybugDB ontology traversal for anatomical entity disambiguation
|
- [ ] ladybugDB ontology traversal for anatomical entity disambiguation
|
||||||
- [ ] GemmaE2B/MedGemma Vietnamese LLM consult (browser WebLLM local OR cloud Vertex AI) with MOH guideline citations
|
- [ ] 3-model LLM consult system: Edge Gemma (browser WebLLM local), Gemini (GCP Vertex AI, orchestration/translation), MedGemma (Modal, clinical deep-reasoning) with MOH guideline citations and cost guarding (<20% MedGemma usage)
|
||||||
- [ ] Circular 46/2018 PDF report generation
|
- [ ] Circular 46/2018 PDF report generation
|
||||||
- [ ] Immutable audit log append (NFR-17) and HITL digital signature gate (NFR-19)
|
- [ ] Immutable audit log append (NFR-17) and HITL digital signature gate (NFR-19)
|
||||||
|
|
||||||
### Non-Functional (Critical)
|
### Non-Functional (Critical)
|
||||||
- [ ] NFR-4: 150 MB idle app bundle
|
- [ ] NFR-4: 150 MB idle app bundle
|
||||||
- [ ] NFR-5: ≤1.5 s inference (on-prem Triton)
|
- [ ] NFR-5: ≤1.5 s inference (on-prem Triton); cloud tiers targeted at <30s median
|
||||||
- [ ] NFR-7: ≤200 ms TTFT token streaming
|
- [ ] NFR-7: ≤200 ms TTFT token streaming; progressive streaming with 25s timeout for cloud responses
|
||||||
- [ ] NFR-8: Fault-tolerant across Wi-Fi drops (state preserved)
|
- [ ] NFR-8: Fault-tolerant across Wi-Fi drops (state preserved)
|
||||||
|
- [ ] NFR-10: Automated Generative Safety Guardrails: <90% verification prohibited, 100% of LLM-generated patient text explanations must pass verification. 3-tier guardrail: prompt rules + BERT edge detection (Tier 1) → Vertex AI safety filters (Tier 2) → RAG-Referee mandatory validation (Tier 3)
|
||||||
|
- [ ] NFR-11: Onboarding ≤ 45 minutes
|
||||||
|
- [ ] NFR-12: Zero-Friction Explainability Integration
|
||||||
|
- [ ] NFR-13: Spatial Layer-Activation Mapping
|
||||||
- [ ] NFR-14: No client-side GPU/neural accelerator required
|
- [ ] NFR-14: No client-side GPU/neural accelerator required
|
||||||
- [ ] NFR-15: Circular 46 EMR compliance
|
- [ ] NFR-15: Circular 46 EMR compliance
|
||||||
- [ ] NFR-16: Air-gapped primary; NFR-16a PoC fallback with redaction/consent/audit
|
- [ ] NFR-16: Air-gapped primary; NFR-16a PoC fallback with redaction/consent/audit for cloud LLM tiers (Gemini + MedGemma)
|
||||||
|
- [ ] NFR-16a: Emergency Cloud Fallback with Redaction (PoC-SCOPE). Cloud LLM API (GCP Vertex AI Gemini + Modal MedGemma) invoked ONLY when browser-side WebLLM (Edge Gemma) is unavailable or orchestrates cloud task. FastAPI Redaction Middleware MUST strip all Decree 13 PII fields before egress. GCP CSEK + 30-day lifecycle + 1-year auto-delete. Audit-log commit + consent before egress. MedGemma usage <20% target with alerting. NFR-16a retired upon PoC sign-off.
|
||||||
- [ ] NFR-17: Immutable audit log
|
- [ ] NFR-17: Immutable audit log
|
||||||
- [ ] NFR-18: 100% LLM text cites MOH protocol via RAG
|
- [ ] NFR-18: 100% LLM text cites MOH protocol via RAG. Mandatory RAG pre-processing for all 3 tiers (not optional tool-calling). RAG-Referee validates MedGemma output.
|
||||||
- [ ] NFR-19: HITL digital signature before FINALIZED/ARCHIVED
|
- [ ] NFR-19: HITL digital signature before FINALIZED/ARCHIVED
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -79,7 +84,8 @@ System_Boundary(hospital_lan, "Hospital LAN (Air-Gapped, ≤10 Mbps)") {
|
|||||||
System_Boundary(k3s_cluster, "K3s Orchestration Cluster") {
|
System_Boundary(k3s_cluster, "K3s Orchestration Cluster") {
|
||||||
System_Boundary(edge_servers, "Application Server Cluster") {
|
System_Boundary(edge_servers, "Application Server Cluster") {
|
||||||
Container(edge_inference, "Edge Inference Service", "FastAPI (Python 3.12, Uvicorn)", "DICOM ingest, 3-step ML pipeline orchestration, Grad-CAM generation, report assembly, consult SSE streaming.")
|
Container(edge_inference, "Edge Inference Service", "FastAPI (Python 3.12, Uvicorn)", "DICOM ingest, 3-step ML pipeline orchestration, Grad-CAM generation, report assembly, consult SSE streaming.")
|
||||||
Container(rag_svc, "RAG & Knowledge Service", "FastAPI (Python 3.12, asyncpg)", "pgvector top-k retrieval, ladybugDB ontology wrapper, GemmaE2B/MedGemma consult route, RAG-Referee validation.")
|
Container(cloud_llm_gateway, "Cloud LLM Gateway", "FastAPI (Python 3.12, Uvicorn)", "Routes orchestration/translation to Gemini (GCP) and clinical deep-reasoning to MedGemma (Modal); NFR-16a redaction, consent, audit enforcement; consult_mode state machine extension; cost guarding for MedGemma usage.")
|
||||||
|
Container(rag_svc, "RAG & Knowledge Service", "FastAPI (Python 3.12, asyncpg)", "pgvector top-k retrieval, ladybugDB ontology wrapper, RAG-Referee validation for MedGemma output, mandatory RAG pre-processing for all tiers.")
|
||||||
Container(audit_svc, "Audit & EMR Service", "Node.js / FastAPI Worker", "Immutable append-only audit events; HL7/FHIR EMR push with outbox retry.")
|
Container(audit_svc, "Audit & EMR Service", "Node.js / FastAPI Worker", "Immutable append-only audit events; HL7/FHIR EMR push with outbox retry.")
|
||||||
Container(api_gw, "API Gateway", "Envoy", "TLS termination, rate limiting, routing, OIDC validation pass-through.")
|
Container(api_gw, "API Gateway", "Envoy", "TLS termination, rate limiting, routing, OIDC validation pass-through.")
|
||||||
Container(auth_svc, "Identity & Access", "Keycloak", "OIDC, RBAC, realm vkist-msk.")
|
Container(auth_svc, "Identity & Access", "Keycloak", "OIDC, RBAC, realm vkist-msk.")
|
||||||
@@ -94,11 +100,12 @@ System_Boundary(hospital_lan, "Hospital LAN (Air-Gapped, ≤10 Mbps)") {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
System_Ext(triton, "Triton Inference Server", "NVIDIA Triton, ONNX/TensorRT, gRPC 8001", "3-step ML pipeline (angle → inflammation → segmentation) + embedding extraction.")
|
System_Ext(triton, "Triton Inference Server", "NVIDIA Triton, ONNX/TensorRT, gRPC 8001", "3-step ML pipeline (angle → inflammation → segmentation) + EmbeddingGemma RAG embeddings.")
|
||||||
System_Ext(emr, "Hospital EMR / HIS", "HL7/FHIR", "Finalized report storage, prescription sync, ground-truth records.")
|
System_Ext(emr, "Hospital EMR / HIS", "HL7/FHIR", "Finalized report storage, prescription sync, ground-truth records.")
|
||||||
System_Ext(pacs, "PACS / Ultrasound Device", "DICOM/C-MOVE", "Image capture and retrieval.")
|
System_Ext(pacs, "PACS / Ultrasound Device", "DICOM/C-MOVE", "Image capture and retrieval.")
|
||||||
System_Ext(gcp_cdn, "GCP CDN Emergency Fallback", "Signed-URL Cloud CDN (ap-southeast1)", "Non-clinical model weight distribution when intranet CDN unreachable.")
|
System_Ext(gcp_cdn, "GCP CDN Emergency Fallback", "Signed-URL Cloud CDN (ap-southeast1)", "Non-clinical model weight distribution when intranet CDN unreachable.")
|
||||||
System_Ext(vertex_ai, "GCP Vertex AI", "GemmaE2B via REST", "PoC-only NFR-16a inference tier; redacted payloads only.")
|
System_Ext(vertex_ai, "GCP Vertex AI (Gemini)", "Gemini via REST", "PoC-only NFR-16a inference tier: orchestration, translation, UI planning; redacted payloads only.")
|
||||||
|
System_Ext(modal_medgemma, "Modal MedGemma", "FastAPI + Transformers (T4 GPU)", "PoC-only NFR-16a clinical deep-reasoning endpoint; redacted payloads with RAG-Referee validation; keep-warm instances for latency budget.")
|
||||||
|
|
||||||
Rel(radiologist, pwa, "Loads scan, reviews grade, finalizes report, views explanations", "HTTPS 443")
|
Rel(radiologist, pwa, "Loads scan, reviews grade, finalizes report, views explanations", "HTTPS 443")
|
||||||
Rel(admin, nginx, "Deploys, monitors, configures", "HTTPS 443 / SSH")
|
Rel(admin, nginx, "Deploys, monitors, configures", "HTTPS 443 / SSH")
|
||||||
@@ -118,9 +125,16 @@ Rel(edge_inference, minio, "DICOM, overlays, reports", "S3 API")
|
|||||||
Rel(edge_inference, auth_svc, "Token introspection", "OIDC")
|
Rel(edge_inference, auth_svc, "Token introspection", "OIDC")
|
||||||
Rel(edge_inference, obs_stack, "/metrics", "HTTP 9090")
|
Rel(edge_inference, obs_stack, "/metrics", "HTTP 9090")
|
||||||
|
|
||||||
|
Rel(edge_inference, cloud_llm_gateway, "Routes cloud consult", "HTTP 8000")
|
||||||
|
Rel(cloud_llm_gateway, vertex_ai, "Gemini proxy (NFR-16a redacted)", "HTTPS 443 / REST")
|
||||||
|
Rel(cloud_llm_gateway, modal_medgemma, "MedGemma proxy (NFR-16a redacted)", "HTTPS 443 / REST")
|
||||||
|
Rel(cloud_llm_gateway, redis, "Consult mode, consent, rate-limit", "TCP 6379")
|
||||||
|
Rel(cloud_llm_gateway, postgres, "Audit log append", "SQL 5432")
|
||||||
|
|
||||||
Rel(rag_svc, postgres, "pgvector HNSW queries", "SQL 5432")
|
Rel(rag_svc, postgres, "pgvector HNSW queries", "SQL 5432")
|
||||||
Rel(rag_svc, redis, "Guideline cache, pub/sub invalidation", "TCP 6379")
|
Rel(rag_svc, redis, "Guideline cache, pub/sub invalidation", "TCP 6379")
|
||||||
Rel(rag_svc, minio, "Guideline PDF ingestion staging", "S3 API")
|
Rel(rag_svc, minio, "Guideline PDF ingestion staging", "S3 API")
|
||||||
|
Rel(rag_svc, edge_inference, "RAG + RAG-Referee results", "HTTP")
|
||||||
|
|
||||||
Rel(audit_svc, postgres, "Appends immutable audit events", "SQL 5432")
|
Rel(audit_svc, postgres, "Appends immutable audit events", "SQL 5432")
|
||||||
Rel(audit_svc, emr, "Finalized report push", "HL7/FHIR")
|
Rel(audit_svc, emr, "Finalized report push", "HL7/FHIR")
|
||||||
@@ -130,7 +144,6 @@ Rel(edge_inference, emr, "Report push (via audit-svc wrapper)", "HL7/FHIR")
|
|||||||
Rel(pwa, pacs, "Direct DICOM capture / C-MOVE", "DICOM")
|
Rel(pwa, pacs, "Direct DICOM capture / C-MOVE", "DICOM")
|
||||||
|
|
||||||
Rel(k3s_cluster, gcp_cdn, "Model weight fetch fallback (non-clinical)", "HTTPS 443 (signed URL)")
|
Rel(k3s_cluster, gcp_cdn, "Model weight fetch fallback (non-clinical)", "HTTPS 443 (signed URL)")
|
||||||
Rel(edge_inference, vertex_ai, "PoC-only cloud consult (NFR-16a redacted)", "HTTPS 443 / REST")
|
|
||||||
Rel(postgres, minio, "Backup checkpoint", "S3 API")
|
Rel(postgres, minio, "Backup checkpoint", "S3 API")
|
||||||
|
|
||||||
@enduml
|
@enduml
|
||||||
@@ -143,8 +156,14 @@ Container communication summary:
|
|||||||
| PWA | NGINX | HTTPS 443 | All API requests |
|
| PWA | NGINX | HTTPS 443 | All API requests |
|
||||||
| NGINX | Envoy | HTTP 8000 | Route upstream |
|
| NGINX | Envoy | HTTP 8000 | Route upstream |
|
||||||
| Envoy | Edge Inference | HTTP 8000 | `/api/*` |
|
| Envoy | Edge Inference | HTTP 8000 | `/api/*` |
|
||||||
|
| Envoy | Cloud LLM Gateway | HTTP 8000 | `/api/cloud-orchestrate`, `/api/cloud-consult` |
|
||||||
| Envoy | RAG Service | HTTP 8001 | `/rag/*` |
|
| Envoy | RAG Service | HTTP 8001 | `/rag/*` |
|
||||||
| Envoy | Keycloak | HTTP 8080 | OIDC validation |
|
| Envoy | Keycloak | HTTP 8080 | OIDC validation |
|
||||||
|
| Edge Inference | Cloud LLM Gateway | HTTP 8000 | Route cloud consult |
|
||||||
|
| Cloud LLM Gateway | Gemini (Vertex AI) | HTTPS 443 / REST | Orchestration, translation, UI planning (NFR-16a) |
|
||||||
|
| Cloud LLM Gateway | MedGemma (Modal) | HTTPS 443 / REST | Clinical deep-reasoning, report finalization (NFR-16a) |
|
||||||
|
| Cloud LLM Gateway | Redis | TCP 6379 | Consult mode, consent, rate-limit |
|
||||||
|
| Cloud LLM Gateway | Postgres | TCP 5432 | Audit log append |
|
||||||
| Edge Inference | Triton | gRPC 8001 | 3-step ML pipeline + embeddings |
|
| Edge Inference | Triton | gRPC 8001 | 3-step ML pipeline + embeddings |
|
||||||
| Edge Inference | Postgres | TCP 5432 | SQL + pgvector HNSW |
|
| Edge Inference | Postgres | TCP 5432 | SQL + pgvector HNSW |
|
||||||
| Edge Inference | Redis | TCP 6379 | Session, rate-limit, consult-mode |
|
| Edge Inference | Redis | TCP 6379 | Session, rate-limit, consult-mode |
|
||||||
@@ -165,23 +184,24 @@ Container communication summary:
|
|||||||
!include <C4/C4_Component>
|
!include <C4/C4_Component>
|
||||||
|
|
||||||
Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") {
|
Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") {
|
||||||
Component(api, "API Controller", "REST + SSE", "/api/analyze, /api/consult/stream")
|
Component(api, "API Controller", "REST + SSE", "/api/analyze, /api/cloud-orchestrate redirect")
|
||||||
Component(stream, "SSE Token Streamer", "FastAPI StreamingResponse", "200 ms TTFT, heart-beat every 30s")
|
Component(stream, "SSE Token Streamer", "FastAPI StreamingResponse", "200 ms TTFT, heart-beat every 30s")
|
||||||
Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub (client-side pre-check)")
|
Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub (client-side pre-check)")
|
||||||
Component(router, "Inference Router", "consult_mode state", "Tier selection: WASM→Triton→Vertex→Templates")
|
Component(router, "Inference Router", "consult_mode state", "Tier selection: Edge Gemma (tier_1) → Gemini (tier_2, orchestration) → MedGemma (tier_3, clinical) → Templates (tier_4)")
|
||||||
Component(breaker, "Circuit Breaker", "pybreaker", "Wrap Triton + EMR calls; fail-open to templates")
|
Component(breaker, "Circuit Breaker", "pybreaker", "Wrap Triton + EMR + Cloud LLM calls; fail-open to templates")
|
||||||
Component(pipeline, "ML Pipeline", "gRPC client", "Angle→Inflammation→Segmentation→Measurement")
|
Component(pipeline, "ML Pipeline", "gRPC client", "Angle→Inflammation→Segmentation→Measurement")
|
||||||
Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay; base64 PNG to PWA")
|
Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay; base64 PNG to PWA")
|
||||||
Component(report, "Report Builder", "WeasyPrint / ReportLab", "Circular 46 PDF; HITL signature gate before FINALIZED")
|
Component(report, "Report Builder", "WeasyPrint / ReportLab", "Circular 46 PDF; HITL signature gate before FINALIZED")
|
||||||
Component(rag, "RAG Service", "pgvector SQL", "Retrieve top-5 MOH chunks; enforce NFR-18 citation")
|
Component(rag, "RAG Service", "pgvector SQL", "Retrieve top-5 MOH chunks; mandatory pre-generation for all tiers; enforce NFR-18 citation")
|
||||||
Component(referee, "RAG-Referee", "BERT classifier", "Reject LLM text if citation confidence < threshold")
|
Component(referee, "RAG-Referee", "BERT classifier", "Reject MedGemma text if citation confidence < threshold; mandatory for tier_3")
|
||||||
Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; refine/clean residual PII; error if unresolvable")
|
Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; refine/clean residual PII; error if unresolvable")
|
||||||
Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override")
|
Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override; cloud_llm_escalation events")
|
||||||
}
|
}
|
||||||
Rel(api, stream, "delegates", "SSE")
|
Rel(api, stream, "delegates", "SSE")
|
||||||
Rel(api, preproc, "validates image", "sync")
|
Rel(api, preproc, "validates image", "sync")
|
||||||
Rel(api, router, "selects tier", "sync")
|
Rel(api, router, "selects tier", "sync")
|
||||||
Rel(router, pipeline, "invokes", "gRPC")
|
Rel(router, pipeline, "invokes", "gRPC")
|
||||||
|
Rel(router, api, "redirects cloud consult", "HTTP")
|
||||||
Rel(router, stream, "fallback text", "SSE")
|
Rel(router, stream, "fallback text", "SSE")
|
||||||
Rel(api, gradcam, "requests overlay", "sync")
|
Rel(api, gradcam, "requests overlay", "sync")
|
||||||
Rel(api, report, "generates PDF", "sync")
|
Rel(api, report, "generates PDF", "sync")
|
||||||
@@ -192,39 +212,27 @@ Rel(api, audit, "writes event", "sync")
|
|||||||
@enduml
|
@enduml
|
||||||
```
|
```
|
||||||
|
|
||||||
### 4.3 Component Diagrams (Tier 3)
|
**Cloud LLM Gateway**
|
||||||
|
|
||||||
**Edge Inference Service (edge-inference-svc)**
|
|
||||||
|
|
||||||
```plantuml
|
```plantuml
|
||||||
@startuml
|
@startuml
|
||||||
!include <C4/C4_Component>
|
!include <C4/C4_Component>
|
||||||
|
|
||||||
Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") {
|
Container_Boundary(cloud_gw, "cloud-llm-gateway (FastAPI)") {
|
||||||
Component(api, "API Controller", "REST + SSE", "/api/analyze, /api/consult/stream")
|
Component(api, "Cloud API Controller", "REST + SSE", "/api/cloud-orchestrate, /api/cloud-consult, /api/cloud-consult/stream")
|
||||||
Component(stream, "SSE Token Streamer", "FastAPI StreamingResponse", "200 ms TTFT, heart-beat every 30s")
|
Component(gateway, "Cloud LLM Router", "task_type matcher", "Routes orchestration/translation → Gemini (tier_2); clinical deep-reasoning → MedGemma (tier_3)")
|
||||||
Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub (client-side pre-check)")
|
Component(consent, "Consent Enforcer", "Redis + NFR-16a checklist", "Validates consent token + redaction manifest before egress")
|
||||||
Component(router, "Inference Router", "consult_mode state", "Tier selection: WASM→Triton→Vertex→Templates")
|
Component(audit, "Cloud Audit Emitter", "PostgreSQL append-only", "egress_consent, egress_response, cloud_llm_escalation events")
|
||||||
Component(breaker, "Circuit Breaker", "pybreaker", "Wrap Triton + EMR calls; fail-open to templates")
|
Component(cost, "Cost Guard", "Redis counter + Prometheus", "Tracks MedGemma usage ratio; alerts if >20% over 24h window")
|
||||||
Component(pipeline, "ML Pipeline", "gRPC client", "Angle→Inflammation→Segmentation→Measurement")
|
Component(referee, "RAG-Referee Trigger", "BERT classifier", "Mandatory validation for MedGemma output before SSE to PWA")
|
||||||
Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay; base64 PNG to PWA")
|
Component(rag, "RAG Pre-processing", "pgvector SQL", "Mandatory top-k injection before any cloud LLM generation")
|
||||||
Component(report, "Report Builder", "WeasyPrint / ReportLab", "Circular 46 PDF; HITL signature gate before FINALIZED")
|
|
||||||
Component(rag, "RAG Service", "pgvector SQL", "Retrieve top-5 MOH chunks; enforce NFR-18 citation")
|
|
||||||
Component(referee, "RAG-Referee", "BERT classifier", "Reject LLM text if citation confidence < threshold")
|
|
||||||
Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; refine/clean residual PII; error if unresolvable")
|
|
||||||
Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override")
|
|
||||||
}
|
}
|
||||||
Rel(api, stream, "delegates", "SSE")
|
Rel(api, consent, "verifies", "sync")
|
||||||
Rel(api, router, "selects tier", "sync")
|
Rel(api, gateway, "routes", "sync")
|
||||||
Rel(router, pipeline, "invokes", "gRPC")
|
Rel(gateway, rag, "injects chunks", "sync")
|
||||||
Rel(router, stream, "fallback text", "SSE")
|
Rel(gateway, referee, "validates output", "sync")
|
||||||
Rel(api, preproc, "validates image", "sync")
|
Rel(gateway, audit, "logs egress", "sync")
|
||||||
Rel(api, gradcam, "requests overlay", "sync")
|
Rel(gateway, cost, "increments counter", "sync")
|
||||||
Rel(api, report, "generates PDF", "sync")
|
|
||||||
Rel(api, rag, "queries MOH", "SQL")
|
|
||||||
Rel(rag, referee, "validates citations", "sync")
|
|
||||||
Rel(api, nlp, "scrubs output", "sync")
|
|
||||||
Rel(api, audit, "writes event", "sync")
|
|
||||||
@enduml
|
@enduml
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -309,13 +317,14 @@ app/
|
|||||||
│ ├── auth.py # Keycloak OIDC validation
|
│ ├── auth.py # Keycloak OIDC validation
|
||||||
│ ├── phi_scrub.py # Microsoft Presidio redaction gate (NFR-16a); refine edge output; error if unresolvable PII
|
│ ├── phi_scrub.py # Microsoft Presidio redaction gate (NFR-16a); refine edge output; error if unresolvable PII
|
||||||
│ └── audit.py # Append-only event emitter
|
│ └── audit.py # Append-only event emitter
|
||||||
├── routers/
|
├── routers/ # FastAPI route handlers (existing + new cloud LLM)
|
||||||
│ ├── analyze.py # POST /api/analyze (sync 3-step pipeline)
|
│ ├── analyze.py # POST /api/analyze (sync 3-step pipeline)
|
||||||
│ ├── consult.py # SSE /api/consult/stream (GemmaE2B/MedGemma + RAG + guardrail session management)
|
|
||||||
│ ├── pacs.py # C-MOVE proxy + DICOM upload
|
│ ├── pacs.py # C-MOVE proxy + DICOM upload
|
||||||
│ ├── emr.py # HL7/FHIR push with outbox
|
│ ├── emr.py # HL7/FHIR push with outbox
|
||||||
│ └── admin.py # Model update, drift review, cache invalidation
|
│ ├── admin.py # Model update, drift review, cache invalidation
|
||||||
├── services/
|
│ ├── cloud_orchestrate.py # POST /api/cloud-orchestrate (Gemini proxy for orchestration/translation)
|
||||||
|
│ └── cloud_consult.py # POST /api/cloud-consult + GET /api/cloud-consult/stream (MedGemma proxy)
|
||||||
|
├── services/ # Business logic modules (existing + new cloud gateway)
|
||||||
│ ├── inference_router.py # consult_mode state machine
|
│ ├── inference_router.py # consult_mode state machine
|
||||||
│ ├── triton_client.py # gRPC with retry decorator
|
│ ├── triton_client.py # gRPC with retry decorator
|
||||||
│ ├── rag.py # pgvector top-k + citation formatter
|
│ ├── rag.py # pgvector top-k + citation formatter
|
||||||
@@ -324,11 +333,12 @@ app/
|
|||||||
│ ├── redaction.py # Presidio AnonymizerEngine; re-verify edge redaction; refine residual PII; error if unresolvable
|
│ ├── redaction.py # Presidio AnonymizerEngine; re-verify edge redaction; refine residual PII; error if unresolvable
|
||||||
│ ├── report.py # Circular 46 PDF + HITL signature
|
│ ├── report.py # Circular 46 PDF + HITL signature
|
||||||
│ ├── ontology.py # ladybugDB C++ bindings wrapper
|
│ ├── ontology.py # ladybugDB C++ bindings wrapper
|
||||||
│ └── audit_writer.py # WAL append
|
│ ├── audit_writer.py # WAL append
|
||||||
|
│ └── cloud_llm_gateway.py # Routes between Gemini (tier_2) and MedGemma (tier_3) based on task_type + consult_mode; NFR-16a consent/redaction/audit enforcement
|
||||||
├── models/
|
├── models/
|
||||||
│ ├── dto.py # Request/response schemas (Pydantic v2)
|
│ ├── dto.py # Request/response schemas (Pydantic v2)
|
||||||
│ ├── domain.py # Case, Session, Grade, Embedding entities
|
│ ├── domain.py # Case, Session, Grade, Embedding entities
|
||||||
│ └── enums.py # ConsultMode, Grade, Tier
|
│ └── enums.py # ConsultMode (tier_1..tier_4), Grade, Tier
|
||||||
└── infra/
|
└── infra/
|
||||||
├── cache.py # Redis client (5 data types only)
|
├── cache.py # Redis client (5 data types only)
|
||||||
├── db.py # SQLAlchemy async engine + pgvector
|
├── db.py # SQLAlchemy async engine + pgvector
|
||||||
@@ -443,22 +453,31 @@ Server-side FastAPI contracts:
|
|||||||
| Method | Path | Auth | Request | Response | Notes |
|
| Method | Path | Auth | Request | Response | Notes |
|
||||||
|--------|------|------|---------|----------|-------|
|
|--------|------|------|---------|----------|-------|
|
||||||
| POST | `/api/analyze` | JWT | multipart DICOM | JSON + Grad-CAM PNG | Sync, ≤1.5 s target |
|
| POST | `/api/analyze` | JWT | multipart DICOM | JSON + Grad-CAM PNG | Sync, ≤1.5 s target |
|
||||||
| GET | `/api/consult/stream` | JWT + consent | SSE text stream | NDJSON chunks | Token streaming ≤200 ms TTFT |
|
| POST | `/api/cloud-orchestrate` | JWT + consent | JSON (task_type, prompt) | JSON text + tier | Gemini proxy; orchestration/translation/UI tasks |
|
||||||
|
| POST | `/api/cloud-consult` | JWT + consent | JSON (task_type, prompt) | JSON text + tier | MedGemma proxy; clinical deep-reasoning |
|
||||||
|
| GET | `/api/cloud-consult/stream` | JWT + consent | query params | SSE text stream | MedGemma streaming; 25s timeout, 30s budget |
|
||||||
| POST | `/api/emr/push` | JWT | case_id | 202 Accepted | Outbox if offline |
|
| POST | `/api/emr/push` | JWT | case_id | 202 Accepted | Outbox if offline |
|
||||||
| POST | `/api/admin/models` | Admin | modelZip | 200 OK | K3s rolling restart |
|
| POST | `/api/admin/models` | Admin | modelZip | 200 OK | K3s rolling restart |
|
||||||
| GET | `/api/rag/citations?q=` | JWT | query string | JSON top-5 chunks | NFR-18 |
|
| GET | `/api/rag/citations?q=` | JWT | query string | JSON top-5 chunks | NFR-18 |
|
||||||
|
| POST | `/api/safety/guardrail-check` | JWT | payload | GuardrailResult | NFR-10 edge guardrail |
|
||||||
|
| GET | `/api/health` | — | — | HealthStatus | Liveness |
|
||||||
|
| GET | `/api/model-registry` | JWT | — | ModelCatalog | Model registry |
|
||||||
|
| POST | `/api/models/register` | Admin | modelZip | RegistrationResult | Model upload |
|
||||||
|
|
||||||
### 6.2 SSE Consult Stream Contract
|
### 6.2 SSE Consult Stream Contract
|
||||||
|
|
||||||
```
|
```
|
||||||
event: token
|
event: token
|
||||||
data: {"text":"The","confidence":0.92}
|
data: {"text":"The","confidence":0.92,"tier":"tier_2"}
|
||||||
|
|
||||||
event: citation
|
event: citation
|
||||||
data: {"source":"MOH guideline 2024 §3.2","page":12}
|
data: {"source":"MOH guideline 2024 §3.2","page":12,"tier":"tier_2"}
|
||||||
|
|
||||||
|
event: tier_change
|
||||||
|
data: {"from":"tier_1","to":"tier_2","reason":"edge_unavailable"}
|
||||||
|
|
||||||
event: done
|
event: done
|
||||||
data: {"tier":"tier_2","latency_ms":340}
|
data: {"tier":"tier_2","latency_ms":340,"model":"gemini-2.5-pro"}
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -475,6 +494,9 @@ data: {"tier":"tier_2","latency_ms":340}
|
|||||||
| Redis | — | OSS | **Use** | Scoped to 5 types; self-hosted |
|
| Redis | — | OSS | **Use** | Scoped to 5 types; self-hosted |
|
||||||
| MinIO | — | OSS | **Use** | S3-compatible; self-hosted |
|
| MinIO | — | OSS | **Use** | S3-compatible; self-hosted |
|
||||||
| Auth | Build (Keycloak) | Auth0/Okta SaaS | **Build** | NFR-16: Keycloak on-prem, no SaaS identity |
|
| Auth | Build (Keycloak) | Auth0/Okta SaaS | **Build** | NFR-16: Keycloak on-prem, no SaaS identity |
|
||||||
|
| Cloud LLM Gateway | Build | — | **Build** | FastAPI service enforcing NFR-16a redaction/consent/audit; routes Gemini/MedGemma based on task_type |
|
||||||
|
| Gemini (Vertex AI) | — | GCP PoC | **Use (NFR-16a)** | Orchestration, translation, UI planning; PoC only with redaction |
|
||||||
|
| MedGemma (Modal) | — | Modal deploy | **Deploy (NFR-16a)** | Clinical deep-reasoning; PoC only with redaction + RAG-Referee |
|
||||||
| EMR integration | Build | Mirth Connect | **Build** | Thin HL7 wrapper FastAPI → EMR; keeps surface area small |
|
| EMR integration | Build | Mirth Connect | **Build** | Thin HL7 wrapper FastAPI → EMR; keeps surface area small |
|
||||||
| CI/CD | Jenkins in K3s | SaaS GitHub Actions | **Build** | Jenkins inside LAN; cloud GitLab via SSH |
|
| CI/CD | Jenkins in K3s | SaaS GitHub Actions | **Build** | Jenkins inside LAN; cloud GitLab via SSH |
|
||||||
| Issue tracking | Self-hosted Jira | Atlassian Cloud | **Build (NFR-16a)** | Cloud VM, compensating controls |
|
| Issue tracking | Self-hosted Jira | Atlassian Cloud | **Build (NFR-16a)** | Cloud VM, compensating controls |
|
||||||
@@ -520,15 +542,20 @@ data: {"tier":"tier_2","latency_ms":340}
|
|||||||
SNOMED-CT knee/hip subset; MSK entity relationships; C++ embedded bindings in FastAPI.
|
SNOMED-CT knee/hip subset; MSK entity relationships; C++ embedded bindings in FastAPI.
|
||||||
|
|
||||||
- [ ] **T3-C RAG service + RAG-Referee** (M)
|
- [ ] **T3-C RAG service + RAG-Referee** (M)
|
||||||
`/rag/query`; top-5 retrieval; BERT classifier to validate LLM citations; reject if threshold fail.
|
`/rag/query`; top-5 retrieval; BERT classifier to validate LLM citations; reject if threshold fail. Mandatory for MedGemma (Tier 3) output; lightweight for Gemini (Tier 2) and Edge Gemma (Tier 1).
|
||||||
|
|
||||||
- [ ] **T3-D Browser WebLLM + Cloud MedGemma LLM endpoints** (L)
|
- [ ] **T3-D 3-Model LLM System: Edge Gemma + Gemini + MedGemma** (L)
|
||||||
Browser: WebLLM (GemmaE2B-Q4) loaded via Service Worker from intranet/GCP CDN; runs in separate WebWorker from CV pipeline.
|
Tier 1 (Edge Gemma): Browser WebLLM (GemmaE2B-Q4) loaded via Service Worker from intranet/GCP CDN; runs in dedicated WebWorker.
|
||||||
Cloud: MedGemma on GCP Vertex AI (NFR-16a governed); FastAPI wrapper with streaming + Decree 13 redaction middleware.
|
Tier 2 (Gemini): GCP Vertex AI (GCP Vertex AI Gemini) for orchestration, translation, UI planning; FastAPI wrapper with streaming + Decree 13 redaction middleware.
|
||||||
Triton: EmbeddingGemma only (768-dim RAG embeddings), no LLM hosting.
|
Tier 3 (MedGemma): Modal-deployed MedGemma large-24b for clinical deep-reasoning and report finalization; FastAPI gateway with keep-warm instances, 30s timeout, RAG-Referee validation.
|
||||||
|
Tier 4: MOH guideline templates (rule-based fallback).
|
||||||
|
Consult mode state machine extended: `tier_1` → `tier_2` → `tier_3` → `tier_4`.
|
||||||
|
|
||||||
- [ ] **T3-E Decree 13 scrubber (client + server)** (M)
|
- [ ] **T3-E Decree 13 scrubber (client + server)** (M)
|
||||||
Client: Dexie.js pre-egress regex. Server: FastAPI middleware for NFR-16a redaction; role-hash tokens.
|
Client: Dexie.js pre-egress regex. Server: FastAPI middleware for NFR-16a redaction; role-hash tokens. Applied to all cloud tiers (Tier 2 + Tier 3).
|
||||||
|
|
||||||
|
- [ ] **T3-F Cloud LLM Gateway** (M)
|
||||||
|
FastAPI service (`/api/cloud-orchestrate`, `/api/cloud-consult`, `/api/cloud-consult/stream`); routes based on `task_type`; enforces consent + redaction + audit before egress; tracks MedGemma usage for <20% cost guard target; extends consult_mode state machine.
|
||||||
|
|
||||||
### Phase 4: Compliance & HITL
|
### Phase 4: Compliance & HITL
|
||||||
- [ ] **T4-A Immutable audit log** (M)
|
- [ ] **T4-A Immutable audit log** (M)
|
||||||
@@ -561,13 +588,13 @@ Deliverables: K3s cluster up, DB VM ready, CI/CD pipeline green, PWA shell live.
|
|||||||
Deliverables: `/api/analyze` end-to-end; Grad-CAM overlay visible in PWA; circuit-breaker handles Triton failure.
|
Deliverables: `/api/analyze` end-to-end; Grad-CAM overlay visible in PWA; circuit-breaker handles Triton failure.
|
||||||
|
|
||||||
### Week 5-6: Phase 3
|
### Week 5-6: Phase 3
|
||||||
Deliverables: RAG queries return MOH citations; GemmaE2B/MedGemma streams Vietnamese explanations; RAG-Referee blocks unmapped LLM text.
|
Deliverables: RAG queries return MOH citations; Edge Gemma (WebLLM) primary, Gemini (Vertex AI) for orchestration/translation, MedGemma (Modal) for clinical deep-reasoning; RAG-Referee validates MedGemma output; consult_mode state machine extended to tier_1→tier_2→tier_3→tier_4.
|
||||||
|
|
||||||
### Week 7: Phase 4
|
### Week 7: Phase 4
|
||||||
Deliverables: Audit log immutable; HITL signature enforces finalization; Circular 46 PDF exports.
|
Deliverables: Audit log immutable with new event types (egress_consent_gemini, egress_consent_medgemma, cloud_llm_escalation); HITL signature enforces finalization; Circular 46 PDF exports; cost guarding (<20% MedGemma usage) enforced.
|
||||||
|
|
||||||
### Week 8: Phase 5 + Sprint Review
|
### Week 8: Phase 5 + Sprint Review
|
||||||
Deliverables: Dashboards, drift monitor, integration tests, demo-ready PWA.
|
Deliverables: Dashboards (tier transitions, MedGemma usage count, latency p50/p99, consent events), drift monitor, integration tests, demo-ready PWA with 3-model routing status.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -73,14 +73,19 @@ Guided by the above requirements and the solution_architect_corpus, the architec
|
|||||||
### 4.1 High-Level Overview
|
### 4.1 High-Level Overview
|
||||||
The system adopts a hybrid edge-cloud (on-premise) architecture with the following layers:
|
The system adopts a hybrid edge-cloud (on-premise) architecture with the following layers:
|
||||||
1. **Edge Inference Layer**: Located on-premise at hospitals/clinics, runs lightweight ML models for real-time DICOM processing (NFR-5,6).
|
1. **Edge Inference Layer**: Located on-premise at hospitals/clinics, runs lightweight ML models for real-time DICOM processing (NFR-5,6).
|
||||||
2. **Micro-Services Layer**: Containerized services orchestrated by K3s (Kubernetes-certified, lightweight distribution designed for edge and resource-constrained production environments). K3s selected over alternatives (Docker Swarm, Nomad, ECS Fargate, Cloud Run) on the following criteria: NFR-16 requires on-premise deployment — eliminating ECS Fargate and Cloud Run (cloud-only platforms). Docker Swarm offers lowest PoC operational cost but is in maintenance mode with no viable scaling path; migration to production-grade orchestration requires complete rewrite of all deployment artifacts. Nomad is operationally viable but lacks ecosystem depth and has less documented migration path to multi-cluster federation. K3s is already production-grade at single-site scale, requires no migration to "production" — it IS the production platform. The scaling path to N hospitals is multi-cluster federation via K3s cluster-api, not platform replacement. All manifests and service definitions are forward-compatible with upstream Kubernetes.
|
2. **3-Tier LLM System**:
|
||||||
3. **Data Layer**: On-premise databases (PostgreSQL + pgvector for relational + vector search, Redis for caching, MinIO for S3-compatible object storage).
|
- **Tier 1 (Edge Gemma)**: Browser/WebLLM (GemmaE2B local in WASM) — primary conversation, local inference, instant response, zero network cost. Orchestrates when to call cloud models via FastAPI middleware.
|
||||||
3. **Embedding Layer**: EmbeddingGemma (768-dim) for RAG retrieval vectors; separate from BERT which is reserved for drift monitoring and RAG-Referee classification only.
|
- **Tier 2 (Gemini)**: GCP Vertex AI (Gemini) — orchestration, translation, UI planning, formatting. Governed by NFR-16a with redaction, consent, and audit logging.
|
||||||
3. **Vector Layer (two-tier)**:
|
- **Tier 3 (MedGemma)**: Modal-deployed MedGemma (large-24b) — clinical deep-reasoning and report finalization. Governed by NFR-16a with redaction, consent, audit logging, and RAG-Referee validation.
|
||||||
|
- **Tier 4 (Templates)**: MOH guideline template responses — rule-based, deterministic, safety floor when all inference tiers fail.
|
||||||
|
3. **Micro-Services Layer**: Containerized services orchestrated by K3s (Kubernetes-certified, lightweight distribution designed for edge and resource-constrained production environments). K3s selected over alternatives (Docker Swarm, Nomad, ECS Fargate, Cloud Run) on the following criteria: NFR-16 requires on-premise deployment — eliminating ECS Fargate and Cloud Run (cloud-only platforms). Docker Swarm offers lowest PoC operational cost but is in maintenance mode with no viable scaling path; migration to production-grade orchestration requires complete rewrite of all deployment artifacts. Nomad is operationally viable but lacks ecosystem depth and has less documented migration path to multi-cluster federation. K3s is already production-grade at single-site scale, requires no migration to "production" — it IS the production platform. The scaling path to N hospitals is multi-cluster federation via K3s cluster-api, not platform replacement. All manifests and service definitions are forward-compatible with upstream Kubernetes.
|
||||||
|
4. **Data Layer**: On-premise databases (PostgreSQL + pgvector for relational + vector search, Redis for caching, MinIO for S3-compatible object storage).
|
||||||
|
5. **Embedding Layer**: EmbeddingGemma (768-dim) for RAG retrieval vectors; separate from BERT which is reserved for drift monitoring and RAG-Referee classification only.
|
||||||
|
6. **Vector Layer (two-tier)**:
|
||||||
- **Hot + Warm: pgvector** (Postgres disk-backed HNSW index, ~15K MOH guideline vectors plus session and case embeddings — serves both real-time RAG during consult and non-real-time analytical queries in a single store. NFR-18, NFR-12. NFR-16 compliant.)
|
- **Hot + Warm: pgvector** (Postgres disk-backed HNSW index, ~15K MOH guideline vectors plus session and case embeddings — serves both real-time RAG during consult and non-real-time analytical queries in a single store. NFR-18, NFR-12. NFR-16 compliant.)
|
||||||
- **Cold: S3 Vectors** (AWS ap-southeast-1, billion-scale archival, cross-facility bootstrap source, disaster recovery rebuild origin — accessed under NFR-16a governance.)
|
- **Cold: S3 Vectors** (AWS ap-southeast-1, billion-scale archival, cross-facility bootstrap source, disaster recovery rebuild origin — accessed under NFR-16a governance.)
|
||||||
5. **Presentation Layer**: Progressive Web App (PWA) built with React, Web Workers, and service workers for offline capability and low memory footprint (NFR-4,14).
|
7. **Presentation Layer**: Progressive Web App (PWA) built with React, Web Workers, and service workers for offline capability and low memory footprint (NFR-4,14).
|
||||||
6. **Observability & Security Layer**: Centralized logging (immutable append-only), monitoring, and audit trails (NFR-10,17).
|
8. **Observability & Security Layer**: Centralized logging (immutable append-only), monitoring, and audit trails (NFR-10,17).
|
||||||
|
|
||||||
### 4.2 Data Flow
|
### 4.2 Data Flow
|
||||||
1. User captures DICOM image via PWA frontend.
|
1. User captures DICOM image via PWA frontend.
|
||||||
@@ -104,13 +109,12 @@ The system adopts a hybrid edge-cloud (on-premise) architecture with the followi
|
|||||||
[3] GCP CDN emergency fallback (asia-southeast1 = Singapore, nearest to Vietnam) — activated ONLY when hospital intranet CDN is confirmed unreachable. GCP backend bucket (`cdn-backend-models-vkist`) wraps a Cloud Storage bucket (`gs://vkist-models-{site_id}`) with Cloud CDN enabled; signed-URL access control (no `allUsers` public exposure); cache mode `CACHE_ALL_STATIC`, default TTL 24h, max TTL 7d (matches model version lifecycle). Service Worker receives a 307 redirect to signed CDN URL on intranet CDN failure; PWA caches the fetched weights locally via Cache Storage as normal. Because model weights contain zero patient data — they are static computational artifacts — the GCP CDN fetch event is committed to the on-prem audit log (case_hash, timestamp, consult_mode = "cdn_fallback") but does NOT trigger the full NFR-16a redaction pipeline. The redaction pipeline applies only to Tier 3a inference calls carrying clinical payloads, not to non-clinical model binary fetches.
|
[3] GCP CDN emergency fallback (asia-southeast1 = Singapore, nearest to Vietnam) — activated ONLY when hospital intranet CDN is confirmed unreachable. GCP backend bucket (`cdn-backend-models-vkist`) wraps a Cloud Storage bucket (`gs://vkist-models-{site_id}`) with Cloud CDN enabled; signed-URL access control (no `allUsers` public exposure); cache mode `CACHE_ALL_STATIC`, default TTL 24h, max TTL 7d (matches model version lifecycle). Service Worker receives a 307 redirect to signed CDN URL on intranet CDN failure; PWA caches the fetched weights locally via Cache Storage as normal. Because model weights contain zero patient data — they are static computational artifacts — the GCP CDN fetch event is committed to the on-prem audit log (case_hash, timestamp, consult_mode = "cdn_fallback") but does NOT trigger the full NFR-16a redaction pipeline. The redaction pipeline applies only to Tier 3a inference calls carrying clinical payloads, not to non-clinical model binary fetches.
|
||||||
- **NFR-4 clarification**: NFR-4's 150MB ceiling covers the core application bundle (JS/TS code, UI assets, runtime state) at idle. Supra-150MB memory consumed by models, caches, and inference buffers is bounded separately under the WASM heap cap (1.5GB) and instrumented independently.
|
- **NFR-4 clarification**: NFR-4's 150MB ceiling covers the core application bundle (JS/TS code, UI assets, runtime state) at idle. Supra-150MB memory consumed by models, caches, and inference buffers is bounded separately under the WASM heap cap (1.5GB) and instrumented independently.
|
||||||
- **Inference Fallback Chain** (LLM consult path only — CV pipeline is always on-prem Triton):
|
- **Inference Fallback Chain** (LLM consult path only — CV pipeline is always on-prem Triton):
|
||||||
- **Tier 0** → Browser WebLLM (GemmaE2B local in WASM, instant, zero network — preferred default when device has ≥3GB RAM and WebGPU available). Consult mode: `browser_local`. Triggered automatically on page load if model weights are cached and WebGPU is detected.
|
- **Tier 1** → Browser WebLLM (Edge GemmaE2B local in WASM, instant, zero network — preferred default when device has ≥3GB RAM and WebGPU available). Consult mode: `tier_1`. Triggered automatically on page load if model weights are cached and WebGPU is detected. Responsible for primary conversation and orchestrating cloud calls.
|
||||||
- **Tier 1** → [REMOVED — Triton does NOT host LLM. Triton hosts CV models + EmbeddingGemma only.]
|
|
||||||
- **Tier 2** → On-prem Triton (hospital GPU node — CV pipeline ONLY: angle → inflammation → segmentation + EmbeddingGemma for RAG embeddings). The LLM consult does NOT fall back to Triton. If browser WebLLM is unavailable, the next tier is cloud.
|
- **Tier 2** → On-prem Triton (hospital GPU node — CV pipeline ONLY: angle → inflammation → segmentation + EmbeddingGemma for RAG embeddings). The LLM consult does NOT fall back to Triton. If browser WebLLM is unavailable, the next tier is cloud.
|
||||||
- **Tier 3a** → GCP Vertex AI (MedGemma via REST API — **PoC ONLY under NFR-16a**). Activated when Tier 0 (browser WebLLM) is unavailable (device unsupported, model not cached, memory exceeded) AND user explicitly consents per-session. All payloads MUST pass Decree 13 PII redaction gate (see §5.4) before leaving hospital boundary. GCP project configured with CSEK, 30-day storage lifecycle, 1-year project deletion timer. Consult mode: `cloud_vertex`.
|
- **Tier 2** → GCP Vertex AI (Gemini — **PoC ONLY under NFR-16a**). Activated when Tier 1 (browser WebLLM) is unavailable AND Edge Gemma orchestrates a cloud call for task types: `orchestration`, `translation`, `ui_planning`, `formatting`. All payloads MUST pass Decree 13 PII redaction gate (see §5.4) before leaving hospital boundary. Consult mode: `tier_2`.
|
||||||
- **Tier 3b** → Cloud LLM Arbiter (GCP Vertex AI MedGemma — **PoC ONLY under NFR-16a, BERT-triggered correction**). Activated when BERT drift monitor (UC-74821) detects semantic impasse, logical contradiction, or contextual hallucination during Socratic dialogue, OR when radiologist explicitly requests "second opinion." Same Vertex AI endpoint as Tier 3a, but with arbiter system prompt + retrieved MOH chunks injected. ALL calls require prior audit-log commit of drift signal + user consent + Decree 13 redaction. Consult mode: `cloud_vertex_arbiter`.
|
- **Tier 3** → Modal MedGemma (clinical deep-reasoning & report finalization — **PoC ONLY under NFR-16a**). Activated when RAG confidence < threshold, radiologist explicitly requests "clinical depth", or Edge Gemma detects complex medical reasoning requiring clinical expertise. Same NFR-16a governance as Tier 2. Consult mode: `tier_3`. Output validated by RAG-Referee before reaching radiologist.
|
||||||
- **Tier 3c** → MOH guideline template responses (rule-based, no generative model, always available, deterministic, cited — safety floor when all inference tiers fail). Consult mode: `templates`.
|
- **Tier 4** → MOH guideline template responses (rule-based, no generative model, always available, deterministic, cited — safety floor when all inference tiers fail). Consult mode: `tier_4`.
|
||||||
- **Circuit Breaker**: PWA Feature Flag Manager (`consult_mode` state machine) attempts tiers sequentially: `browser_local` → `cloud_vertex` → `cloud_vertex_arbiter` → `templates`. Emits tier-transition events to on-prem immutable audit log (NFR-17). Tier 3a/3b transitions require consent token + redaction manifest committed *before* the HTTP call. CDN fallback events (model weight distribution only) log lighter audit entry: no redaction required, no consent required (weights carry no PII).
|
- **Circuit Breaker**: PWA Feature Flag Manager (`consult_mode` state machine) attempts tiers sequentially: `tier_1` → `tier_2` → `tier_3` → `tier_4`. Emits tier-transition events to on-prem immutable audit log (NFR-17). Tier 2/3 transitions require consent token + redaction manifest committed *before* the HTTP call. CDN fallback events (model weight distribution only) log lighter audit entry: no redaction required, no consent required (weights carry no PII).
|
||||||
- **GCP CDN configuration** (emergency distribution path only):
|
- **GCP CDN configuration** (emergency distribution path only):
|
||||||
- Cloud Storage bucket: `gs://vkist-models-{site_id}` — Uniform bucket-level access, no `allUsers` public exposure.
|
- Cloud Storage bucket: `gs://vkist-models-{site_id}` — Uniform bucket-level access, no `allUsers` public exposure.
|
||||||
- Signed-URL generation: FastAPI generates time-limited signed URLs (signed by project service account private key, CSEK-managed) with a 15-minute expiry; returned to PWA as a `307` redirect on intranet CDN failure.
|
- Signed-URL generation: FastAPI generates time-limited signed URLs (signed by project service account private key, CSEK-managed) with a 15-minute expiry; returned to PWA as a `307` redirect on intranet CDN failure.
|
||||||
@@ -133,7 +137,7 @@ The system adopts a hybrid edge-cloud (on-premise) architecture with the followi
|
|||||||
- **JWT session state** (TTL 1h, matches token expiry): validates active sessions without Postgres roundtrip; coalesced look-aside with single-flight mutex to prevent thundering herd on session-miss.
|
- **JWT session state** (TTL 1h, matches token expiry): validates active sessions without Postgres roundtrip; coalesced look-aside with single-flight mutex to prevent thundering herd on session-miss.
|
||||||
- **MOH guideline chunks** (TTL 7d): version-key invalidation triggered by Postgres NOTIFY on guideline ingestion events; cache key = `guideline:{version}:{chunk_id}`, full key-space retired on version bump, no TTL races.
|
- **MOH guideline chunks** (TTL 7d): version-key invalidation triggered by Postgres NOTIFY on guideline ingestion events; cache key = `guideline:{version}:{chunk_id}`, full key-space retired on version bump, no TTL races.
|
||||||
- **DICOM metadata per session** (TTL 12h, per-session scope): avoids repeated Postgres lookups of DICOM headers during the 3-step ML pipeline; evicted after session completion.
|
- **DICOM metadata per session** (TTL 12h, per-session scope): avoids repeated Postgres lookups of DICOM headers during the 3-step ML pipeline; evicted after session completion.
|
||||||
- **Circuit-breaker / consult_mode state** (TTL 2h, matches session): consult_mode enum (tier_1/tier_2/tier_3a/tier_3b) keyed by session_hash. Written atomically on tier transitions. Read by: FastAPI routing layer, PWA (via SSE status push), Prometheus metrics exporter. Justifies Redis over Postgres: sub-ms reads required, writes are frequent (per BERT token stream during Socratic dialogue), data is volatile (expires with session), and multiple FastAPI workers behind NGINX need shared visibility — in-process memory cannot cross worker boundaries.
|
- **Circuit-breaker / consult_mode state** (TTL 2h, matches session): consult_mode enum (tier_1/tier_2/tier_3/tier_4) keyed by session_hash. Written atomically on tier transitions. Read by: FastAPI routing layer, PWA (via SSE status push), Prometheus metrics exporter. Justifies Redis over Postgres: sub-ms reads required, writes are frequent (per BERT token stream during Socratic dialogue), data is volatile (expires with session), and multiple FastAPI workers behind NGINX need shared visibility — in-process memory cannot cross worker boundaries.
|
||||||
- **Rate-limit counters** (TTL 30s, sliding window): INCR on consult request entry. Self-clearing via EXPIRE. Prevents Triton GPU saturation under concurrent burst from multiple radiologists. Atomic Redis operation; Postgres row-lock alternative creates write amplification on hot path.
|
- **Rate-limit counters** (TTL 30s, sliding window): INCR on consult request entry. Self-clearing via EXPIRE. Prevents Triton GPU saturation under concurrent burst from multiple radiologists. Atomic Redis operation; Postgres row-lock alternative creates write amplification on hot path.
|
||||||
- **Not cached in Redis** (served from primary store):
|
- **Not cached in Redis** (served from primary store):
|
||||||
- Audit log: append-only, NFR-17 immutability requirement — no intermediate cache layer.
|
- Audit log: append-only, NFR-17 immutability requirement — no intermediate cache layer.
|
||||||
@@ -248,9 +252,9 @@ The system adopts a hybrid edge-cloud (on-premise) architecture with the followi
|
|||||||
- **Application**: NeMo Guardrails and GuardrailsAI require substantial runtime scaffolding, rule engines, and WASM-compatible bindings that do not exist in the browser ecosystem at the model inference layer. The client-side cannot run a NeMo Rail or GuardrailsAI pipeline inside a WebWorker alongside WebLLM. Therefore, the edge guardrail tier relies on two lightweight mechanisms:
|
- **Application**: NeMo Guardrails and GuardrailsAI require substantial runtime scaffolding, rule engines, and WASM-compatible bindings that do not exist in the browser ecosystem at the model inference layer. The client-side cannot run a NeMo Rail or GuardrailsAI pipeline inside a WebWorker alongside WebLLM. Therefore, the edge guardrail tier relies on two lightweight mechanisms:
|
||||||
1. **Robust prompt-based rules**: Hard system-prompt constraints injected at every consult turn. Boundaries: no diagnosis outside MOH protocol scope; mandatory citation for every clinical assertion; no free-text echoing of PII even if user inputs it; immediate refusal to follow instructions that attempt prompt injection or jailbreak; tone and scope locked to clinical decision-support.
|
1. **Robust prompt-based rules**: Hard system-prompt constraints injected at every consult turn. Boundaries: no diagnosis outside MOH protocol scope; mandatory citation for every clinical assertion; no free-text echoing of PII even if user inputs it; immediate refusal to follow instructions that attempt prompt injection or jailbreak; tone and scope locked to clinical decision-support.
|
||||||
2. **BERT hallucination / mal-intention detection**: Each candidate LLM token stream (or per-turn summary) is evaluated by a local Transformers.js BERT classifier running in a dedicated `guardrail.worker.ts` WebWorker. The classifier scores for: (a) hallucination probability (factual grounding vs retrieved MOH context); (b) mal-intention / prompt-injection indicators (user-context leaking, instruction override attempts, jailbreak markers); (c) scope violation (claims outside synovitis/MSK MOH guidelines).
|
2. **BERT hallucination / mal-intention detection**: Each candidate LLM token stream (or per-turn summary) is evaluated by a local Transformers.js BERT classifier running in a dedicated `guardrail.worker.ts` WebWorker. The classifier scores for: (a) hallucination probability (factual grounding vs retrieved MOH context); (b) mal-intention / prompt-injection indicators (user-context leaking, instruction override attempts, jailbreak markers); (c) scope violation (claims outside synovitis/MSK MOH guidelines).
|
||||||
- **Session termination and cloud mitigation**: If BERT edge guardrail returns a violation score above threshold on any axis → FastAPI immediately terminates the WebLLM session (`stream.close()`), logs the trigger event to immutable audit, and opens a new mitigated consult session toward Tier 3a (cloud Vertex AI MedGemma). Mitigation path: the same user query + retrieved MOH chunks (already computed) + edge-detected-offense log are forwarded under NFR-16a governance (consent token confirmed, Decree 13 redaction re-applied, audit-log commit before egress). The radiologist sees: "Edge model restricted — escalating to cloud consult (redacted)."
|
- **Session termination and cloud mitigation**: If BERT edge guardrail returns a violation score above threshold on any axis → FastAPI immediately terminates the WebLLM session (`stream.close()`), logs the trigger event to immutable audit, and routes to the appropriate cloud tier based on `task_type`: orchestration/translation tasks route to Tier 2 (Gemini), clinical deep-reasoning/report finalization routes to Tier 3 (MedGemma). Mitigation path: the same user query + retrieved MOH chunks (already computed) + edge-detected-offense log are forwarded under NFR-16a governance (consent token confirmed, Decree 13 redaction re-applied, audit-log commit before egress). The radiologist sees: "Edge model restricted — escalating to cloud consult (redacted)."
|
||||||
- **WebWorker isolation**: `guardrail.worker.ts` runs in a separate WebWorker from `cv.worker.ts` (LiteRT) and `llm.worker.ts` (WebLLM). No shared WASM memory between workers. IndexedDB caches artifacts but is never injected into LLM context. Unload order on memory pressure: `llm.worker.ts` → `guardrail.worker.ts` → `cv.worker.ts`.
|
- **WebWorker isolation**: `guardrail.worker.ts` runs in a separate WebWorker from `cv.worker.ts` (LiteRT) and `llm.worker.ts` (WebLLM). No shared WASM memory between workers. IndexedDB caches artifacts but is never injected into LLM context. Unload order on memory pressure: `llm.worker.ts` → `guardrail.worker.ts` → `cv.worker.ts`.
|
||||||
- **Cloud model guardrail**: Cloud Vertex AI MedGemma (Tier 3a/3b) uses natively supported Vertex AI Model Garden safety filters (tuned for healthcare) plus the same server-side FastAPI BERT RAG-Referee gate. No additional client-side framework required.
|
- **Cloud model guardrail**: Cloud models use the server-side FastAPI BERT RAG-Referee gate as the primary safety mechanism. Tier 2 (Gemini) uses Vertex AI Model Garden safety filters (tuned for healthcare). Tier 3 (MedGemma) output is additionally validated by RAG-Referee before reaching the radiologist. No additional client-side framework required.
|
||||||
|
|
||||||
### 5.8 Edge Data Hygiene: Anonymization, Redaction & Server-Side Ground-Check
|
### 5.8 Edge Data Hygiene: Anonymization, Redaction & Server-Side Ground-Check
|
||||||
- **Pattern**: Edge-to-Server Redaction Pipeline with Ground-Truth Verification
|
- **Pattern**: Edge-to-Server Redaction Pipeline with Ground-Truth Verification
|
||||||
@@ -269,22 +273,23 @@ The system adopts a hybrid edge-cloud (on-premise) architecture with the followi
|
|||||||
### 5.9 RAG as Essential Pipeline Step (Non-Optional)
|
### 5.9 RAG as Essential Pipeline Step (Non-Optional)
|
||||||
- **Pattern**: Mandatory RAG Pre-Processing for All LLM Consult Paths
|
- **Pattern**: Mandatory RAG Pre-Processing for All LLM Consult Paths
|
||||||
- **Source**: NFR-18 (100% LLM text cites MOH protocol); NFR-12 (zero-friction explainability)
|
- **Source**: NFR-18 (100% LLM text cites MOH protocol); NFR-12 (zero-friction explainability)
|
||||||
- **Application**: RAG retrieval is NOT an optional tool the LLM may choose to invoke. It is a mandatory pipeline step that executes before any LLM generation, for both edge (browser WebLLM) and cloud (Vertex AI MedGemma) tiers.
|
- **Application**: RAG retrieval is NOT an optional tool the LLM may choose to invoke. It is a mandatory pipeline step that executes before any LLM generation, for all three-tier systems: Edge Gemma (browser WebLLM), Gemini (GCP Vertex AI), and MedGemma (Modal).
|
||||||
- **Pipeline order**:
|
- **Pipeline order**:
|
||||||
1. User query arrives (post-redaction)
|
1. User query arrives (post-redaction)
|
||||||
2. `rag.query()` retrieves top-k MOH guideline chunks from pgvector HNSW
|
2. `rag.query()` retrieves top-k MOH guideline chunks from pgvector HNSW
|
||||||
3. Retrieved chunks + strict system prompt ("You are a clinical decision-support assistant. Answer ONLY using the provided MOH guidelines. Cite chunk IDs. Do not speculate.") → injected into LLM context
|
3. Retrieved chunks + strict system prompt ("You are a clinical decision-support assistant. Answer ONLY using the provided MOH guidelines. Cite chunk IDs. Do not speculate.") → injected into LLM context
|
||||||
4. LLM generates grounded, cited response
|
4. LLM generates grounded, cited response
|
||||||
5. `referee.validate()` (BERT RAG-Referee) checks citation correctness, logical cohesion, factual contestant status
|
5. `referee.validate()` (BERT RAG-Referee) checks citation correctness, logical cohesion, factual contestant status → mandatory for MedGemma (Tier 3) output; optional but recommended for Gemini (Tier 2); lightweight validation for Edge Gemma (Tier 1)
|
||||||
6. If any axis fails → reject → fallback to MOH template
|
6. If any axis fails → reject → fallback to MOH template
|
||||||
- **Rationale**: If RAG were merely a tool call, a compromised or under-performing edge LLM could choose to skip retrieval and generate ungrounded text, violating NFR-18. Mandatory RAG pre-processing closes this vector.
|
- **Rationale**: If RAG were merely a tool call, a compromised or under-performing edge LLM could choose to skip retrieval and generate ungrounded text, violating NFR-18. Mandatory RAG pre-processing closes this vector.
|
||||||
|
|
||||||
### 5.10 Tool-Calling Semantics for Edge and Cloud LLMs
|
### 5.10 Tool-Calling Semantics for Edge and Cloud LLMs
|
||||||
- **Pattern**: Function-Calling as Convenience Layer Over Mandatory RAG
|
- **Pattern**: Function-Calling as Convenience Layer Over Mandatory RAG
|
||||||
- **Source**: Gemma Functions (Google DeepMind) + Vertex AI Function Calling
|
- **Source**: Gemma Functions (Google DeepMind) + Vertex AI Function Calling + Modal MedGemma Function Calling
|
||||||
- **Application**:
|
- **Application**:
|
||||||
- **Edge LLM (GemmaE2B via WebLLM)**: Supports Gemma Functions. The RAG retrieval is exposed as a function tool (`retrieve_moh_guidelines(query, top_k)`). HOWEVER, this tool is pre-invoked by the host pipeline per §5.9 before the first LLM token is generated. The function-calling interface is available for subsequent turns of Socratic dialogue where the model may need additional retrieval mid-conversation (e.g., user asks "what about contraindications?" → mid-turn retrieval). The host pipeline enforces the mandatory RAG call on turn 0; subsequent per-turn retrieval is model-initiated via function calling.
|
- **Edge LLM (Edge Gemma via WebLLM)**: Supports Gemma Functions. The RAG retrieval is exposed as a function tool (`retrieve_moh_guidelines(query, top_k)`). HOWEVER, this tool is pre-invoked by the host pipeline per §5.9 before the first LLM token is generated. The function-calling interface is available for subsequent turns of Socratic dialogue where the model may need additional retrieval mid-conversation (e.g., user asks "what about contraindications?" → mid-turn retrieval). The host pipeline enforces the mandatory RAG call on turn 0; subsequent per-turn retrieval is model-initiated via function calling.
|
||||||
- **Cloud LLM (MedGemma on Vertex AI)**: Native Vertex AI Function Calling is enabled. Same semantics: server-side FastAPI injects top-k MOH chunks before first generation; model may request further retrieval via function call during multi-turn consult.
|
- **Cloud LLM Tier 2 (Gemini on Vertex AI)**: Native Vertex AI Function Calling is enabled for orchestration/translation tasks. Same semantics: server-side FastAPI injects top-k MOH chunks before first generation; model may request further retrieval via function call during multi-turn consult.
|
||||||
|
- **Cloud LLM Tier 3 (MedGemma on Modal)**: Native function-calling interface for clinical reasoning tasks. Server-side FastAPI injects top-k MOH chunks and RAG-Referee validation results before first generation; model may request further retrieval via function call during multi-turn clinical consult.
|
||||||
- **Why not pure tool-calling**: Pure tool-calling delegates the decision to retrieve to the model. NFR-18 requires 100% citation; delegating retrieval to model choice creates a non-zero probability of unsupported generation. Mandatory pre-injection + optional per-turn tool call is the safer contract.
|
- **Why not pure tool-calling**: Pure tool-calling delegates the decision to retrieve to the model. NFR-18 requires 100% citation; delegating retrieval to model choice creates a non-zero probability of unsupported generation. Mandatory pre-injection + optional per-turn tool call is the safer contract.
|
||||||
|
|
||||||
### 5.11 Output Filtering
|
### 5.11 Output Filtering
|
||||||
@@ -304,6 +309,11 @@ The system adopts a hybrid edge-cloud (on-premise) architecture with the followi
|
|||||||
- `edge_guardrail_violation`: session_hash, violation_axis (hallucination|mal_intention|scope_breach), bert_score, user_query_hash, mitigation_target (cloud|template).
|
- `edge_guardrail_violation`: session_hash, violation_axis (hallucination|mal_intention|scope_breach), bert_score, user_query_hash, mitigation_target (cloud|template).
|
||||||
- `rag_retrieval_event`: session_hash, query_hash, retrieved_chunk_ids, latency_ms.
|
- `rag_retrieval_event`: session_hash, query_hash, retrieved_chunk_ids, latency_ms.
|
||||||
- `referee_decision`: session_hash, axis_results (attribution|cohesion|contestant), overall_pass, fallback_triggered.
|
- `referee_decision`: session_hash, axis_results (attribution|cohesion|contestant), overall_pass, fallback_triggered.
|
||||||
|
- `egress_consent_gemini`: session_hash, user_id, task_type, redaction_hash — before Gemini cloud call.
|
||||||
|
- `egress_consent_medgemma`: session_hash, user_id, task_type, redaction_hash — before MedGemma cloud call.
|
||||||
|
- `egress_response_gemini`: session_hash, task_type, status, ts — after Gemini response.
|
||||||
|
- `egress_response_medgemma`: session_hash, task_type, status, ts — after MedGemma response.
|
||||||
|
- `cloud_llm_escalation`: session_hash, from_tier, to_tier, reason (reroute, referee_fail, token_limit) — tracks Gemini→MedGemma or any cross-tier escalation.
|
||||||
- **Audit events for this layer**: `redaction_edge_manifest` (client), `redaction_server_refine` (server-side cleaning pass), `redaction_failure` (server unable to clean → error), `egress_consent` + `egress_redact_manifest` (before cloud call, NFR-16a).
|
- **Audit events for this layer**: `redaction_edge_manifest` (client), `redaction_server_refine` (server-side cleaning pass), `redaction_failure` (server unable to clean → error), `egress_consent` + `egress_redact_manifest` (before cloud call, NFR-16a).
|
||||||
- **Retention**: Audit events retained indefinitely (append-only, no UPDATE/DELETE). Raw LLM token streams retained for 90 days in MinIO (compressed, AES-256 at rest) for drift analysis. After 90 days, streams are deleted; only aggregated `referee_decision` and `rag_retrieval_event` summaries remain.
|
- **Retention**: Audit events retained indefinitely (append-only, no UPDATE/DELETE). Raw LLM token streams retained for 90 days in MinIO (compressed, AES-256 at rest) for drift analysis. After 90 days, streams are deleted; only aggregated `referee_decision` and `rag_retrieval_event` summaries remain.
|
||||||
- **BERT drift monitor telemetry**: edge BERT classifier score distributions exported to Prometheus; alert if false-positive rate on hallucination detection exceeds 5% or false-negative rate exceeds 0.1% (clinical safety threshold).
|
- **BERT drift monitor telemetry**: edge BERT classifier score distributions exported to Prometheus; alert if false-positive rate on hallucination detection exceeds 5% or false-negative rate exceeds 0.1% (clinical safety threshold).
|
||||||
|
|||||||
110
proj_level_reading/DEMO_EXP/Functional_Requirements.md
Normal file
110
proj_level_reading/DEMO_EXP/Functional_Requirements.md
Normal file
@@ -0,0 +1,110 @@
|
|||||||
|
# DEMO_EXP — Functional Requirements (Demo-FRs)
|
||||||
|
|
||||||
|
> **Status:** Draft — Demo & Testing Only
|
||||||
|
> **Date:** June 25, 2026
|
||||||
|
> **Engineer:** Đạt Trần Tiến (Daves Tran)
|
||||||
|
> **Scope:** These Demo-FRs are experimental add-ons layered onto the existing VKIST FR architecture. They are **not** production FRs and shall be tracked separately from `FR_Engineer_DB_Mobile.csv`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Demo-FR-01: "Morning Cohort Ritual" — Synchronized Patient List Entry
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
| :--- | :--- |
|
||||||
|
| **Demo-FR ID** | Demo-FR-01 |
|
||||||
|
| **Title** | Morning Cohort Ritual — Synchronized Patient List Entry |
|
||||||
|
| **Platform** | Mobile Web (PWA) |
|
||||||
|
| **Component** | Patient Responsibility Dashboard (extends FR-26) |
|
||||||
|
| **User Profile** | UP5 (Diagnostic Radiologist) |
|
||||||
|
| **Interaction** | User-to-System |
|
||||||
|
| **Base FR** | FR-26 (`UC-48377`: View Patient List with Clinical Summary) |
|
||||||
|
| **Sprint** | Sprint 3 (The Collaborative Workspace) |
|
||||||
|
| **Priority** | Demo / Proof-of-Concept |
|
||||||
|
| **Precondition** | User has valid authentication and FR-26 patient roster is loaded. |
|
||||||
|
| **Postcondition** | Each patient card in the roster displays a context strip showing: (1) the last collective action taken on the case (with actor + timestamp), (2) which other roles are currently active on this case, and (3) any pending action required from the radiologist. No extra navigation is required. |
|
||||||
|
| **Trigger** | When the user opens the Patient Responsibility Dashboard (`UC-48377`), the system enriches each patient card entry with synchronized context data drawn from EMR sign-off logs, scan session state, and PT scheduling records. |
|
||||||
|
| **Stimulus** | User taps "My Patients" / "View Patient List" button on the Mobile Web PWA dashboard. |
|
||||||
|
| **System Response** | System retrieves the authenticated user's patient roster AND enriches each entry with: (a) last-signal metadata (who acted, when, what action), (b) active-role indicators (Surgeon? PT? both?), (c) pending-action flags (e.g., "Awaiting your grading on Scan #7"). Renders enriched cards in-place. |
|
||||||
|
| **NFR Constraints** | Zero extra clicks — context strip rendered inline on existing cards. All data passes through Decree 13 scrub layer. Compatible with zero-GPU fallback rendering. |
|
||||||
|
|
||||||
|
**Motivation:**
|
||||||
|
Transforms a sterile data lookup into a collective handoff moment. The radiologist opens the list and instantly understands *where they fit in the care chain* — who else is working on this case, what was the last decision, and what is pending their action.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Demo-FR-02: "Agreement Moment" — Sign-Off Cascade
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
| :--- | :--- |
|
||||||
|
| **Demo-FR ID** | Demo-FR-02 |
|
||||||
|
| **Title** | Agreement Moment — Synchronized Sign-Off Cascade |
|
||||||
|
| **Platform** | Mobile Web (PWA) |
|
||||||
|
| **Component** | Report Finalization Layer (extends FR-25) |
|
||||||
|
| **User Profile** | UP5 (Diagnostic Radiologist), Surgeon, PT, Patient/Caregiver |
|
||||||
|
| **Interaction** | User-to-System (multi-role) |
|
||||||
|
| **Base FR** | FR-25 / `UC-92006` (Finalize & Sign Electronic Record) |
|
||||||
|
| **Sprint** | Sprint 3 (The Collaborative Workspace) |
|
||||||
|
| **Priority** | Demo / Proof-of-Concept |
|
||||||
|
| **Precondition** | Radiologist has completed grading review and is ready to sign the diagnostic report. All 4 roles have active accounts in the system. |
|
||||||
|
| **Postcondition** | Upon radiologist's cryptographic seal: (1) Surgeon's dashboard receives a real-time pulse notification with finalized report summary, (2) PT's read-only view is updated with the new PT protocol card, (3) Patient/Caregiver portal receives a plain-language notification of the result, (4) EMR log entry is created. All 3 downstream notifications fire within the same synchronization window. |
|
||||||
|
| **Trigger** | Radiologist taps "Finalize & Sign" on the diagnostic report screen. |
|
||||||
|
| **Stimulus** | User (UP5) confirms sign-off action. System displays a single confirmation: "This will notify the care team and patient. Proceed?" |
|
||||||
|
| **System Response** | Upon confirmation: (1) cryptographic seal applied, (2) EMR log entry created (`UC-02423`), (3) Surgeon dashboard pulse notification dispatched via existing event bus, (4) PT protocol card delivered to PT workspace via existing FR-30 send pipeline, (5) plain-language patient notification dispatched via existing FR-19 push infrastructure. All fires within <= 2 seconds of sign-off. |
|
||||||
|
| **NFR Constraints** | Zero extra workflow steps — sign-off is a single tap. Notification infrastructure reuses existing FR-19/FR-30 pipelines. All data passes through Decree 13 scrub. No external APIs — all sync over local K3s cluster. |
|
||||||
|
|
||||||
|
**Motivation:**
|
||||||
|
Elevates the act of signing from a bureaucratic checkbox into a *synchronized collective event*. The radiologist's single action of clinical judgment becomes the trigger that unlocks the next phase of care for every other participant. The radiologist *feels* that their signature has weight and consequence.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Demo-FR-03: "Daily Progress Ritual" — Synchronized PT–Patient Journal
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
| :--- | :--- |
|
||||||
|
| **Demo-FR ID** | Demo-FR-03 |
|
||||||
|
| **Title** | Daily Progress Ritual — Synchronized PT–Patient Journal |
|
||||||
|
| **Platform** | Mobile Web (PWA) |
|
||||||
|
| **Component** | Patient Education / Care Logic Module (extends FR-28, FR-29) |
|
||||||
|
| **User Profile** | UP8 (MSK Patient & Family Caregiver), UP6 (Physical Therapist) |
|
||||||
|
| **Interaction** | User-to-System (dual-user, time-shifted) |
|
||||||
|
| **Base FR** | FR-28 (CREATE patient treatment journal), FR-29 (UPDATE patient treatment journal) |
|
||||||
|
| **Sprint** | Sprint 4 (Patient-Facing PWA) |
|
||||||
|
| **Priority** | Demo / Proof-of-Concept |
|
||||||
|
| **Precondition** | Patient treatment journal has been initialized and is within the active treatment monitoring period. |
|
||||||
|
| **Postcondition** | Patient journal entry is saved with timestamp and trend indicator. PT dashboard for that patient is updated with a soft indicator showing the latest entry timestamp, pain level delta, and trend direction. Patient receives positive visual feedback upon submission. |
|
||||||
|
| **Trigger** | Patient submits a journal update (pain slider adjustment, symptom checkbox, or free-text entry). |
|
||||||
|
| **Stimulus** | Patient interacts with the journal entry UI on their mobile device. |
|
||||||
|
| **System Response** | (1) Journal entry is saved with timestamp and anonymized patient ID, (2) Decree 13 scrub layer validates no PII in entry, (3) PT dashboard for that patient is updated with a soft indicator: latest entry timestamp, pain level delta (e.g., "from 7 to 3"), trend arrow, (4) Patient sees positive ritual feedback (e.g., progress arc animation, culturally appropriate encouragement text). |
|
||||||
|
| **NFR Constraints** | Journal data already scrubbed before PT view (Decree 13). Ritual UI overlay is a lightweight animation layer on existing journal UI — no new screens. Zero-GPU compatible. No open-ended communication loops — data flows one-way (patient -> PT), gated by existing FR-30 send pipeline. |
|
||||||
|
|
||||||
|
**Motivation:**
|
||||||
|
Transforms a repetitive, lonely self-reporting task into a *shared daily ritual*. The patient feels *seen* — their effort is acknowledged, not just logged. The PT enters the session already connected to the patient's lived experience, turning a generic protocol execution into a continuation of a shared story of progress.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Demo-FR-04: "Journey Mirror" — Longitudinal Synchronized Progress View
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
| :--- | :--- |
|
||||||
|
| **Demo-FR ID** | Demo-FR-04 |
|
||||||
|
| **Title** | Journey Mirror — Longitudinal Synchronized Progress View |
|
||||||
|
| **Platform** | Mobile Web (PWA) |
|
||||||
|
| **Component** | Patient Portal / History Module (extends FR-16, FR-31, FR-12) |
|
||||||
|
| **User Profile** | UP8 (MSK Patient & Family Caregiver), UP5 (Diagnostic Radiologist), UP7 (Surgeon/Orthopedic) |
|
||||||
|
| **Interaction** | User-to-System (dual-lens, same data) |
|
||||||
|
| **Base FR** | FR-16 (TRA CUU lich su kham benh), FR-31 (INTERPRET diagnostic report from Clinic to Patient), FR-12 (SYNCHRONIZE hardware-adaptive musculoskeletal models) |
|
||||||
|
| **Sprint** | Sprint 4-5 (Patient-Facing PWA -> Feedback Pipeline & Hardening) |
|
||||||
|
| **Priority** | Demo / Proof-of-Concept |
|
||||||
|
| **Precondition** | Patient has >= 2 historical scan sessions in the system. FR-16 history data is accessible. FR-31 plain-language interpretation layer is active. |
|
||||||
|
| **Postcondition** | Both clinician and patient can view the same longitudinal timeline: (1) Clinician lens shows scan dates, AI grades, GradCAM overlays, EMR sign-off timestamps, surgical decisions, (2) Patient lens shows the same timeline nodes with 3D visit thumbnails and plain-language captions. Timeline updates are synchronized — when a clinician adds a new scan, the patient's mirror view reflects it. |
|
||||||
|
| **Trigger** | User (either role) opens the "My Journey" / "Patient Journey" timeline view. |
|
||||||
|
| **Stimulus** | User navigates to the longitudinal history view from their dashboard. |
|
||||||
|
| **System Response** | (1) System retrieves full patient visit history from FR-16 history store, (2) Renders timeline nodes: each node populated from scan session data (FR-25/UC-48376), signed report metadata (FR-25/UC-92006), and plain-language interpretation (FR-31), (3) Clinician view: renders with AI grades, GradCAM overlay thumbnails, EMR timestamps, (4) Patient view: renders with zero-GPU 3D sprite-sheet thumbnails (FR-12) and plain-language captions, (5) Both views reference the same timeline data — synchronized by patient ID and visit date. |
|
||||||
|
| **NFR Constraints** | Zero-GPU fallback renders sprite-sheet thumbnails on legacy devices (FR-12). Decree 13: clinician view shows anonymized ID; patient view shows only their own record. All rendering over local K3s cluster, no external CDN. |
|
||||||
|
|
||||||
|
**Motivation:**
|
||||||
|
Creates a **shared narrative** of the patient's healing journey. Both the clinician and the patient are telling the *same story* — the clinician through clinical precision, the patient through plain language and visual empathy. The timeline becomes a ritual object that both parties return to, reinforcing trust and shared understanding across the entire care arc.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*End of Functional Requirements — Demo-FRs (4 total)*
|
||||||
100
proj_level_reading/DEMO_EXP/Motivation.md
Normal file
100
proj_level_reading/DEMO_EXP/Motivation.md
Normal file
@@ -0,0 +1,100 @@
|
|||||||
|
# DEMO_EXP — Motivation: "Synchronized Ritual" Concept
|
||||||
|
|
||||||
|
> **Status:** Draft — Concept Note for Demo Session
|
||||||
|
> **Date:** June 25, 2026
|
||||||
|
> **Engineer:** Đạt Trần Tiến (Daves Tran)
|
||||||
|
> **Context:** VKIST-PILOT MSK Pilot Workspace (Sprint 1-6, June 2 - September 2, 2026)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The UX Concept: What Is a "Synchronized Ritual"?
|
||||||
|
|
||||||
|
In UX and digital design, a **Synchronized Ritual** is the intentional creation of shared digital experiences or synchronous interactions that connect users **across time and space**. It takes a basic, habitual app action and elevates it into an emotionally resonant, collective, or deeply personalized experience.
|
||||||
|
|
||||||
|
A ritual is not just a repeated action. A *synchronized* ritual is one where:
|
||||||
|
- **Multiple users** feel connected to the **same event** at the same moment, even if they are physically apart
|
||||||
|
- **Past, present, and future** are visible in a single view — the user feels the *arc* of what they are part of
|
||||||
|
- A **basic action** (open a list, tap a button) carries **emotional weight** beyond its functional outcome
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Why This Concept Matters for VKIST Specifically
|
||||||
|
|
||||||
|
PILOT's user research reveals **three structural loneliness problems** that Synchronized Ritual directly addresses:
|
||||||
|
|
||||||
|
### Problem 1: The Isolated Radiologist
|
||||||
|
Radiologists work in **dark, quiet reading rooms**, detached from direct patient contact and from the downstream clinicians who consume their reports. Their work is invisible until a Surgeon or PT reads it — sometimes hours or days later.
|
||||||
|
|
||||||
|
> "They are the vital gatekeepers of clinical safety. Engagement and UX research should be focused entirely on invisible workflow optimization." — User Research Result, UP5 profile
|
||||||
|
|
||||||
|
**Synchronized Ritual response:** Create moments where the radiologist's action *immediately ripples outward*. When they sign a report, they see — in real time — that the Surgeon's dashboard has pulsed, the PT's session has updated, and the patient has been notified. The radiologist *feels* their work is part of a collective machine.
|
||||||
|
|
||||||
|
### Problem 2: The Anxious, Information-Starved Patient
|
||||||
|
Patients and their caregivers are **anxious and overwhelmed**, often turning to dangerous folk remedies (leaf-wrapping, aggressive manual adjustment) because no one has translated their scan into understandable terms. The hospital consultation is brief; the patient leaves confused.
|
||||||
|
|
||||||
|
> "If the AI is expertly presented as a high-tech, highly objective authority that visually validates their human doctor's hurried diagnosis, it can significantly and immediately increase institutional trust." — User Research Result, UP8 profile
|
||||||
|
|
||||||
|
**Synchronized Ritual response:** Transform the AI-generated report into a **shared artifact** — the doctor's clinical precision and the patient's emotional understanding, synchronized in one view. The patient sees their own healing arc as a narrative of progress, not a stack of confusing documents.
|
||||||
|
|
||||||
|
### Problem 3: The Information-Siloed PT
|
||||||
|
Physical Therapists receive only **brief text prescriptions** — they lack visibility into the radiologist's scan findings, the surgeon's protocol, or the patient's subjective experience. They execute treatment in semi-blindness.
|
||||||
|
|
||||||
|
> "Vietnamese Physiotherapists struggle to accurately target internal tissue pathologies during therapy because doctors rarely share full digital DICOM imaging data down the chain, providing only brief text prescription sheets, leading to semi-blind therapeutic execution." — User Research Result, UP6 profile
|
||||||
|
|
||||||
|
**Synchronized Ritual response:** Synchronize the PT's treatment session with the radiologist's scan finding and the surgeon's protocol. The PT enters the session already connected to the full clinical chain — not just a one-page prescription.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. The Four Demo-FRs: How Ritual Maps to Existing FR Scope
|
||||||
|
|
||||||
|
Each Demo-FR is an **add-on layer** on top of existing Functional Requirements. No new backend pipelines are required — the rituals reuse data and infrastructure that already exists or is already planned.
|
||||||
|
|
||||||
|
| Demo-FR | Ritual Moment | What Becomes Shared | Base FR(s) |
|
||||||
|
| :--- | :--- | :--- | :--- |
|
||||||
|
| **01: Morning Cohort Ritual** | Opening the patient list | The *collective context* of each case — who else is working on it, what was last decided | FR-26, FR-25/UC-92006 |
|
||||||
|
| **02: Agreement Moment** | Signing a diagnostic report | The *act of clinical judgment* ripples to Surgeon, PT, Patient simultaneously | FR-25/UC-92006, FR-19, FR-30 |
|
||||||
|
| **03: Daily Progress Ritual** | Patient logging symptoms | The *daily narrative* of recovery, shared between patient and PT before the session begins | FR-28, FR-29, FR-30, FR-20 |
|
||||||
|
| **04: Journey Mirror** | Viewing the patient's history | The *longitudinal arc* of healing, seen through two synchronized lenses (clinical + patient) | FR-16, FR-31, FR-12 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. The Engineering Rationale: Why This Is Not "Just UX Polish"
|
||||||
|
|
||||||
|
The Synchronized Ritual concept is grounded in PILOT's **hard NFRs and constraints**, not in abstract UX theory:
|
||||||
|
|
||||||
|
| PILOT NFR | How Demo-FRs Respect It |
|
||||||
|
| :--- | :--- |
|
||||||
|
| **Zero Workflow Friction** — no extra clicks, no extra screens | All rituals are embedded in existing actions: opening a list, signing a report, logging a symptom. The ritual is the *enriched rendering* of data that already exists. |
|
||||||
|
| **Decree 13 / Data Privacy** — strict scrubbing | Ritual data (timestamps, actor names, status chips) is already available in scrubbed form in EMR logs and scan session metadata. No new PII is introduced. |
|
||||||
|
| **Hardware Heterogeneity** — low-spec phones, old GPUs | Ritual UI uses existing rendering pipelines: patient cards, status chips, sprite-sheet thumbnails. Zero-GPU fallback is already required by FR-12. |
|
||||||
|
| **Air-Gapped Infrastructure** — no cloud, no external APIs | All synchronization happens over the local K3s cluster / FastAPI backend. The "sync" in Synchronized Ritual is *local data propagation*, not cloud-based real-time networking. |
|
||||||
|
| **Sprint 6 Deadline** — only 6 sprints | Demo-FRs are achieved by **recomposing existing data** (EMR logs, scan sessions, journal entries) into new UI patterns — not by building new data pipelines or ML models. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. The Stakeholder Narrative: How to Present This
|
||||||
|
|
||||||
|
The Synchronized Ritual Demo is not a feature demo. It is a **values demo**.
|
||||||
|
|
||||||
|
> "We are not showing you what the system does. We are showing you how it makes every person in this care chain feel connected to each other's work. That is the difference between a tool that processes data and a platform that honors the clinical relationship."
|
||||||
|
|
||||||
|
**The demo opens with Demo-FR-02 (Agreement Moment)** — it has the highest emotional impact: one tap, four lives informed simultaneously. Then peel back to show how each layer (01, 03, 04) builds on the same principle.
|
||||||
|
|
||||||
|
**The closing argument:**
|
||||||
|
> "All of this runs locally at PILOT. No data leaves the hospital. No extra clicks were added to anyone's workflow. And yet — the radiologist who signs a report at 14:30 knows that at 14:30, a surgeon in the OR, a PT in the treatment room, and a patient at home all received a piece of that same moment. That is what we mean by Synchronized Ritual."
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Risks & Guardrails
|
||||||
|
|
||||||
|
| Risk | Mitigation |
|
||||||
|
| :--- | :--- |
|
||||||
|
| Ritual UI perceived as "extra work" by busy clinicians | All rituals are **implicit** — they appear in existing views (list cards, sign-off confirmation, journal screen). No new buttons or new screens. |
|
||||||
|
| Patient-facing rituals trigger anxiety (e.g., "pain went up" notification) | Ritual feedback uses **positive framing** — progress arcs, encouragement text, trend arrows. No alarmist language. |
|
||||||
|
| Synchronized data introduces a privacy concern | All ritual data is already present in EMR logs and scan sessions. The ritual layer only *re-frames* existing data — it does not aggregate new data sources. |
|
||||||
|
| Demo feels "theatrical" rather than substantive | Demo scripts use **realistic synthetic data** (e.g., "Case #VN-2048", "Dr. Nguyen", "pain 7->3") and avoid cartoonish UI flourishes. The emotional weight comes from the *logic of connection*, not decorative animation. |
|
||||||
|
| Clinician rejection of "patient-facing" features | Demo-FR-02 and Demo-FR-04 are presented from the **clinician's perspective first** — the ritual is about *their* sense of impact, not about "making patients happy." |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*End of Motivation — Synchronized Ritual Concept for PILOT Demo Session*
|
||||||
167
proj_level_reading/DEMO_EXP/Use_Cases.md
Normal file
167
proj_level_reading/DEMO_EXP/Use_Cases.md
Normal file
@@ -0,0 +1,167 @@
|
|||||||
|
# DEMO_EXP — Use Cases (Demo UCs)
|
||||||
|
|
||||||
|
> **Status:** Draft — Demo & Testing Only
|
||||||
|
> **Date:** June 25, 2026
|
||||||
|
> **Engineer:** Đạt Trần Tiến (Daves Tran)
|
||||||
|
> **Scope:** These Use Cases describe the Demo-FR interactions for the "Synchronized Ritual" experiments. They are **not** production UCs and shall be tracked separately from `FR_25_UC_SPEC.md`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## UC-48377: View Patient List with Clinical Summary
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
| :--- | :--- |
|
||||||
|
| **UC-ID** | UC-48377 |
|
||||||
|
| **Title [Verb + Noun]** | View Patient List with Clinical Summary |
|
||||||
|
| **Actor** | UP5 (Diagnostic Radiologist) |
|
||||||
|
| **System Actor** | System (Backend) |
|
||||||
|
| **Date Added** | June 25, 2026 |
|
||||||
|
| **Engineer** | Đạt Trần Tiến (Daves Tran) |
|
||||||
|
| **Interaction** | User-to-System |
|
||||||
|
| **Platform** | Mobile Web (PWA) |
|
||||||
|
| **FR Link** | FR-26 (VIEW PATIENT LIST WITH RESPONSIBILITY DETAILS) |
|
||||||
|
| **Demo-FR Extensions** | Demo-FR-01 (Morning Cohort Ritual — context strip), Demo-FR-02 (Agreement Moment — sign-off cascade trigger), Demo-FR-04 (Journey Mirror — longitudinal timeline entry point) |
|
||||||
|
|
||||||
|
**Goal:**
|
||||||
|
Access a structured list of patients under the user's clinical responsibility to monitor active diagnostic cases and treatment tracks.
|
||||||
|
|
||||||
|
**Preconditions:**
|
||||||
|
- User has valid authentication credentials and active session with the Mobile Web PWA.
|
||||||
|
- User has at least one patient case assigned under their clinical responsibility scope.
|
||||||
|
|
||||||
|
**Postconditions (Success State):**
|
||||||
|
- Patient roster list is fully rendered in the mobile viewport.
|
||||||
|
- Each list entry contains anonymized patient ID, scan type, diagnostic status, prescription summary, and workflow stage.
|
||||||
|
- *(Demo extension)* Each entry also shows synchronized context: last action actor + timestamp, active-role indicators, and pending-action flags.
|
||||||
|
|
||||||
|
**Stimulus:**
|
||||||
|
The user clicks the "View Patient List" button in the Mobile Web PWA dashboard.
|
||||||
|
|
||||||
|
**System Response:**
|
||||||
|
The system retrieves the authenticated user's patient roster and displays a scrollable list, each entry showing patient anonymized ID, most recent scan type, current diagnostic status (e.g., AI-graded, pending review, finalized), prescription summary, and active workflow stage.
|
||||||
|
*(Demo extension)* The system enriches each card with synchronized context data drawn from EMR sign-off logs, scan session state, and PT scheduling records.
|
||||||
|
|
||||||
|
**Verbose Form:**
|
||||||
|
The use case 'View Patient List with Clinical Summary' defines a User-to-System interaction where the Diagnostic Radiologist (Rad) aims to access a structured list of patients under the user's clinical responsibility to monitor active diagnostic cases and treatment tracks. This workflow is triggered when the user clicks the "View Patient List" button in the Mobile Web PWA dashboard, causing the system to respond by providing the system retrieves the authenticated user's patient roster and displays a scrollable list, each entry showing patient anonymized ID, most recent scan type, current diagnostic status, prescription summary, and active workflow stage.
|
||||||
|
|
||||||
|
**Main Success Scenario (Happy Path):**
|
||||||
|
1. Diagnostic Radiologist taps the "My Patients" navigation chip on the PWA dashboard.
|
||||||
|
2. System validates the active JWT token and queries the backend patient roster endpoint.
|
||||||
|
3. System retrieves the authenticated user's assigned patient list with associated clinical metadata (scan type, AI grade status, prescription summary, workflow stage).
|
||||||
|
4. System renders a scrollable, paginated list on the mobile viewport, with each card showing anonymized patient ID, most recent scan modality, current diagnostic status tag, and active treatment workflow indicator.
|
||||||
|
5. *(Demo extension)* System enriches each card entry with synchronized context strip: last action metadata, active-role indicators, and pending-action flags.
|
||||||
|
6. System enables pull-to-refresh to reload the roster without full page reload.
|
||||||
|
|
||||||
|
**Alternative & Exception Flows:**
|
||||||
|
- **Exception A: Empty Patient Roster** — If the authenticated user has no assigned patients, the system displays a placeholder state with a "No patients assigned" message and a contact administrator prompt.
|
||||||
|
- **Exception B: Network Timeout / Session Expiry** — If the backend request times out or the session token is invalid, the system displays a non-blocking connection error banner with a retry action.
|
||||||
|
|
||||||
|
**Demo Scenario:**
|
||||||
|
> "The radiologist opens the app on their mobile device. Instead of a flat list of anonymized IDs, each patient card now shows a thin contextual bar: 'Case #VN-2048 — Surgeon Dr. Nguyen reviewed protocol 14:22 today | PT session scheduled 16:00 | Awaiting your grading on Scan #7'. The radiologist immediately understands where they fit in the collective care chain, without clicking into anything."
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## UC-48378: Filter Patient List by Clinical Status
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
| :--- | :--- |
|
||||||
|
| **UC-ID** | UC-48378 |
|
||||||
|
| **Title [Verb + Noun]** | Filter Patient List by Clinical Status |
|
||||||
|
| **Actor** | UP5 (Diagnostic Radiologist) |
|
||||||
|
| **System Actor** | System (Backend) |
|
||||||
|
| **Date Added** | June 25, 2026 |
|
||||||
|
| **Engineer** | Đạt Trần Tiến (Daves Tran) |
|
||||||
|
| **Interaction** | User-to-System |
|
||||||
|
| **Platform** | Mobile Web (PWA) |
|
||||||
|
| **FR Link** | FR-26 (VIEW PATIENT LIST WITH RESPONSIBILITY DETAILS) |
|
||||||
|
| **Demo-FR Extensions** | Demo-FR-01 (Morning Cohort Ritual — context strip preserved across filters), Demo-FR-02 (Agreement Moment — filtering before sign-off) |
|
||||||
|
|
||||||
|
**Goal:**
|
||||||
|
Narrow the patient roster to a specific diagnostic or treatment sub-population to reduce cognitive load during high-volume clinical sessions.
|
||||||
|
|
||||||
|
**Preconditions:**
|
||||||
|
- Full patient roster is loaded and displayed in the PWA viewport (`UC-48377`).
|
||||||
|
- At least one filter dimension is available (status, scan type, date range).
|
||||||
|
|
||||||
|
**Postconditions (Success State):**
|
||||||
|
- List viewport is updated to show only matching patient entries.
|
||||||
|
- Active filter chips are rendered to indicate current constraint state.
|
||||||
|
- Context strip from `UC-48377` is preserved on all filtered entries.
|
||||||
|
|
||||||
|
**Stimulus:**
|
||||||
|
The user applies a filter (by status, scan type, or date range) to the currently displayed patient list.
|
||||||
|
|
||||||
|
**System Response:**
|
||||||
|
The system immediately updates the list to show only patients matching the selected criteria, with visible filter chips indicating active constraints. Context strips remain intact on all visible entries.
|
||||||
|
|
||||||
|
**Verbose Form:**
|
||||||
|
The use case 'Filter Patient List by Clinical Status' defines a User-to-System interaction where the Diagnostic Radiologist (Rad) aims to narrow the patient roster to a specific diagnostic or treatment sub-population to reduce cognitive load during high-volume clinical sessions. This workflow is triggered when the user applies a filter (by status, scan type, or date range) to the currently displayed patient list, causing the system to respond by providing the system immediately updates the list to show only patients matching the selected criteria, with visible filter chips indicating active constraints.
|
||||||
|
|
||||||
|
**Main Success Scenario (Happy Path):**
|
||||||
|
1. Diagnostic Radiologist taps a filter chip or dropdown control above the patient list (e.g., "Pending Review", "AI-Graded", "Sup-Long", "Last 7 Days").
|
||||||
|
2. System captures the selected filter parameters and re-queries the patient roster endpoint with applied constraints.
|
||||||
|
3. System updates the rendered list in-place, preserving the scroll position and context strips on all matching entries.
|
||||||
|
4. System displays active filter chips above the list to provide clear visual feedback of current constraints.
|
||||||
|
5. System enables the user to stack additional filters or clear all filters with a single "Reset" action.
|
||||||
|
|
||||||
|
**Alternative & Exception Flows:**
|
||||||
|
- **Exception A: No Results After Filter** — If the applied filter yields zero matching patients, the system displays a "No patients match current filters" empty state with a direct "Clear Filters" action button.
|
||||||
|
- **Exception B: Partial Filter Failure** — If one of multiple selected filters fails to apply (e.g., server timeout on a date-range constraint), the system retains the last successfully filtered list state and highlights the failed filter chip in an error state.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## UC-48379: Navigate to Patient Detail Record
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
| :--- | :--- |
|
||||||
|
| **UC-ID** | UC-48379 |
|
||||||
|
| **Title [Verb + Noun]** | Navigate to Patient Detail Record |
|
||||||
|
| **Actor** | UP5 (Diagnostic Radiologist) |
|
||||||
|
| **System Actor** | System (Backend) |
|
||||||
|
| **Date Added** | June 25, 2026 |
|
||||||
|
| **Engineer** | Đạt Trần Tiến (Daves Tran) |
|
||||||
|
| **Interaction** | User-to-System |
|
||||||
|
| **Platform** | Mobile Web (PWA) |
|
||||||
|
| **FR Link** | FR-26 (VIEW PATIENT LIST WITH RESPONSIBILITY DETAILS) |
|
||||||
|
| **Demo-FR Extensions** | Demo-FR-01 (Morning Cohort Ritual — navigation from enriched list), Demo-FR-02 (Agreement Moment — detail view before sign-off), Demo-FR-04 (Journey Mirror — detail view as timeline entry point) |
|
||||||
|
|
||||||
|
**Goal:**
|
||||||
|
Drill into a specific patient's consolidated clinical workspace to review full diagnostic history, AI-generated overlays, and treatment context before rendering a final judgment.
|
||||||
|
|
||||||
|
**Preconditions:**
|
||||||
|
- Patient roster list is rendered in the PWA viewport (`UC-48377`).
|
||||||
|
- Patient record contains at least one diagnostic session entry accessible by the authenticated user.
|
||||||
|
|
||||||
|
**Postconditions (Success State):**
|
||||||
|
- Full patient workspace is loaded, showing consolidated diagnostic history, AI overlays, prescription summaries, and active treatment protocol notes.
|
||||||
|
- All displayed data is Decree 13-scrubbed (no raw PII visible to the user).
|
||||||
|
- *(Demo extension)* Longitudinal timeline entry point is accessible from the detail view, enabling navigation to Demo-FR-04 Journey Mirror.
|
||||||
|
|
||||||
|
**Stimulus:**
|
||||||
|
The user taps a specific patient entry from the list (filtered or unfiltered).
|
||||||
|
|
||||||
|
**System Response:**
|
||||||
|
The system navigates to the patient's full clinical workspace, displaying longitudinal diagnostic history, AI-segmented anatomical overlays with GradCAM heatmaps, prescription and diagnosis summaries, and current treatment protocol notes.
|
||||||
|
|
||||||
|
**Verbose Form:**
|
||||||
|
The use case 'Navigate to Patient Detail Record' defines a User-to-System interaction where the Diagnostic Radiologist (Rad) aims to drill into a specific patient's consolidated clinical workspace to review full diagnostic history, AI-generated overlays, and treatment context before rendering a final judgment. This workflow is triggered when the user taps a specific patient entry from the list (filtered or unfiltered), causing the system to respond by providing the system navigates to the patient's full clinical workspace, displaying longitudinal diagnostic history, AI-segmented anatomical overlays with GradCAM heatmaps, prescription and diagnosis summaries, and current treatment protocol notes.
|
||||||
|
|
||||||
|
**Main Success Scenario (Happy Path):**
|
||||||
|
1. Diagnostic Radiologist taps a patient card entry from the displayed roster list.
|
||||||
|
2. System sends a detail-record fetch request to the backend with the patient session identifier.
|
||||||
|
3. System retrieves longitudinal diagnostic history, including prior scan metadata, AI-generated segmentation overlays, GradCAM heatmaps, prescription summaries, and active treatment protocol notes.
|
||||||
|
4. System validates that all data has been through the Decree 13 scrubbing layer before rendering.
|
||||||
|
5. System renders the consolidated patient workspace, displaying a timeline of prior scans, the most recent AI-graded overlays, and current prescription and diagnosis summaries in a single rapid-consumption view.
|
||||||
|
6. *(Demo extension)* System enables navigation to the Journey Mirror timeline view (Demo-FR-04) from within the patient workspace.
|
||||||
|
7. System enables the radiologist to tap any historical scan entry to load the full diagnostic workspace (`UC-48376`).
|
||||||
|
|
||||||
|
**Alternative & Exception Flows:**
|
||||||
|
- **Exception A: Patient Record Locked or Restricted** — If the patient record is flagged with an access restriction (e.g., different responsible clinician), the system displays an "Access Restricted" dialog with a request access action rather than exposing partial data.
|
||||||
|
- **Exception B: Incomplete Scrubbing Detected** — If the scrubbing validation layer detects residual PII tokens in the payload, the system halts rendering and triggers a server-side re-scrubbing cycle before presenting the record.
|
||||||
|
|
||||||
|
**Demo Scenario:**
|
||||||
|
> "The radiologist taps a patient card. The system loads the full workspace: a timeline of past scans on the left, the most recent AI overlay in the center, prescription summary below. The radiologist reviews the GradCAM heatmap, sees the Surgeon's treatment protocol note from 14:22 today, and is ready to finalize — all without leaving the mobile viewport."
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*End of Use Cases — Demo UCs (3 total)*
|
||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Binary file not shown.
File diff suppressed because it is too large
Load Diff
Binary file not shown.
File diff suppressed because it is too large
Load Diff
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1,510 +0,0 @@
|
|||||||
<!-- image -->
|
|
||||||
|
|
||||||
## Architectural design
|
|
||||||
|
|
||||||
## Objectives
|
|
||||||
|
|
||||||
The objective of this chapter is to introduce the concepts of software architecture and architectural design. When you have read the chapter, you will:
|
|
||||||
|
|
||||||
- ■ understand why the architectural design of software is important;
|
|
||||||
- ■ understand the decisions that have to be made about the system architecture during the architectural design process;
|
|
||||||
- ■ have been introduced to the idea of architectural patterns, well-tried ways of organizing system architectures, which can be reused in system designs;
|
|
||||||
- ■ know the architectural patterns that are often used in different types of application system, including transaction processing systems and language processing systems.
|
|
||||||
|
|
||||||
## Contents
|
|
||||||
|
|
||||||
- 6.1 Architectural design decisions
|
|
||||||
- 6.2 Architectural views
|
|
||||||
- 6.3 Architectural patterns
|
|
||||||
- 6.4 Application architectures
|
|
||||||
|
|
||||||
Architectural design is concerned with understanding how a system should be organized and designing the overall structure of that system. In the model of the software development process, as shown in Chapter 2, architectural design is the first stage in the software design process. It is the critical link between design and requirements engineering, as it identifies the main structural components in a system and the relationships between them. The output of the architectural design process is an architectural model that describes how the system is organized as a set of communicating components.
|
|
||||||
|
|
||||||
In agile processes, it is generally accepted that an early stage of the development process should be concerned with establishing an overall system architecture. Incremental development of architectures is not usually successful. While refactoring components in response to changes is usually relatively easy, refactoring a system architecture is likely to be expensive.
|
|
||||||
|
|
||||||
To help you understand what I mean by system architecture, consider Figure 6.1. This shows an abstract model of the architecture for a packing robot system that shows the components that have to be developed. This robotic system can pack different kinds of object. It uses a vision component to pick out objects on a conveyor, identify the type of object, and select the right kind of packaging. The system then moves objects from the delivery conveyor to be packaged. It places packaged objects on another conveyor. The architectural model shows these components and the links between them.
|
|
||||||
|
|
||||||
In practice, there is a significant overlap between the processes of requirements engineering and architectural design. Ideally, a system specification should not include any design information. This is unrealistic except for very small systems. Architectural decomposition is usually necessary to structure and organize the specification. Therefore, as part of the requirements engineering process, you might propose an abstract system architecture where you associate groups of system functions or features with large-scale components or sub-systems. You can then use this decomposition to discuss the requirements and features of the system with stakeholders.
|
|
||||||
|
|
||||||
You can design software architectures at two levels of abstraction, which I call architecture in the small and architecture in the large :
|
|
||||||
|
|
||||||
1. Architecture in the small is concerned with the architecture of individual programs. At this level, we are concerned with the way that an individual program is decomposed into components. This chapter is mostly concerned with program architectures.
|
|
||||||
2. Architecture in the large is concerned with the architecture of complex enterprise systems that include other systems, programs, and program components. These enterprise systems are distributed over different computers, which may be owned and managed by different companies. I cover architecture in the large in Chapters 18 and 19, where I discuss distributed systems architectures.
|
|
||||||
|
|
||||||
Figure 6.1 The architecture of a packing robot control system Software architecture is important because it affects the performance, robustness, distributability, and maintainability of a system (Bosch, 2000). As Bosch discusses, individual components implement the functional system requirements. The nonfunctional requirements depend on the system architecture-the way in which these components are organized and communicate. In many systems, non-functional requirements are also influenced by individual components, but there is no doubt that the architecture of the system is the dominant influence.
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
Bass et al. (2003) discuss three advantages of explicitly designing and documenting software architecture:
|
|
||||||
|
|
||||||
1. Stakeholder communication The architecture is a high-level presentation of the system that may be used as a focus for discussion by a range of different stakeholders.
|
|
||||||
2. System analysis Making the system architecture explicit at an early stage in the system development requires some analysis. Architectural design decisions have a profound effect on whether or not the system can meet critical requirements such as performance, reliability, and maintainability.
|
|
||||||
3. Large-scale reuse A model of a system architecture is a compact, manageable description of how a system is organized and how the components interoperate. The system architecture is often the same for systems with similar requirements and so can support large-scale software reuse. As I explain in Chapter 16, it may be possible to develop product-line architectures where the same architecture is reused across a range of related systems.
|
|
||||||
|
|
||||||
Hofmeister et al. (2000) propose that a software architecture can serve firstly as a design plan for the negotiation of system requirements, and secondly as a means of structuring discussions with clients, developers, and managers. They also suggest that it is an essential tool for complexity management. It hides details and allows the designers to focus on the key system abstractions.
|
|
||||||
|
|
||||||
System architectures are often modeled using simple block diagrams, as in Figure 6.1. Each box in the diagram represents a component. Boxes within boxes indicate that the component has been decomposed to sub-components. Arrows mean that data and or control signals are passed from component to component in the direction of the arrows. Y ou can see many examples of this type of architectural model in Booch's software architecture catalog (Booch, 2009).
|
|
||||||
|
|
||||||
Block diagrams present a high-level picture of the system structure, which people from different disciplines, who are involved in the system development process, can readily understand. However, in spite of their widespread use, Bass et al. (2003) dislike informal block diagrams for describing an architecture. They claim that these informal diagrams are poor architectural representations, as they show neither the type of the relationships among system components nor the components' externally visible properties.
|
|
||||||
|
|
||||||
The apparent contradictions between practice and architectural theory arise because there are two ways in which an architectural model of a program is used:
|
|
||||||
|
|
||||||
1. As a way of facilitating discussion about the system design A high-level architectural view of a system is useful for communication with system stakeholders and project planning because it is not cluttered with detail. Stakeholders can relate to it and understand an abstract view of the system. They can then discuss the system as a whole without being confused by detail. The architectural model identifies the key components that are to be developed so managers can start assigning people to plan the development of these systems.
|
|
||||||
2. As a way of documenting an architecture that has been designed The aim here is to produce a complete system model that shows the different components in a system, their interfaces, and their connections. The argument for this is that such a detailed architectural description makes it easier to understand and evolve the system.
|
|
||||||
|
|
||||||
Block diagrams are an appropriate way of describing the system architecture during the design process, as they are a good way of supporting communications between the people involved in the process. In many projects, these are often the only architectural documentation that exists. However, if the architecture of a system is to be thoroughly documented then it is better to use a notation with well-defined semantics for architectural description. However, as I discuss in Section 6.2, some people think that detailed documentation is neither useful, nor really worth the cost of its development.
|
|
||||||
|
|
||||||
■
|
|
||||||
|
|
||||||
## 6.1 Architectural design decisions
|
|
||||||
|
|
||||||
Architectural design is a creative process where you design a system organization that will satisfy the functional and non-functional requirements of a system. Because it is a creative process, the activities within the process depend on the type of system being developed, the background and experience of the system architect, and the specific requirements for the system. It is therefore useful to think of architectural design as a series of decisions to be made rather than a sequence of activities.
|
|
||||||
|
|
||||||
During the architectural design process, system architects have to make a number of structural decisions that profoundly affect the system and its development process. Based on their knowledge and experience, they have to consider the following fundamental questions about the system:
|
|
||||||
|
|
||||||
1. Is there a generic application architecture that can act as a template for the system that is being designed?
|
|
||||||
2. How will the system be distributed across a number of cores or processors?
|
|
||||||
3. What architectural patterns or styles might be used?
|
|
||||||
4. What will be the fundamental approach used to structure the system?
|
|
||||||
5. How will the structural components in the system be decomposed into subcomponents?
|
|
||||||
6. What strategy will be used to control the operation of the components in the system?
|
|
||||||
7. What architectural organization is best for delivering the non-functional requirements of the system?
|
|
||||||
8. How will the architectural design be evaluated?
|
|
||||||
9. How should the architecture of the system be documented?
|
|
||||||
|
|
||||||
Although each software system is unique, systems in the same application domain often have similar architectures that reflect the fundamental concepts of the domain. For example, application product lines are applications that are built around a core architecture with variants that satisfy specific customer requirements. When designing a system architecture, you have to decide what your system and broader application classes have in common, and decide how much knowledge from these application architectures you can reuse. I discuss generic application architectures in Section 6.4 and application product lines in Chapter 16.
|
|
||||||
|
|
||||||
For embedded systems and systems designed for personal computers, there is usually only a single processor and you will not have to design a distributed architecture for the system. However, most large systems are now distributed systems in which the system software is distributed across many different computers. The choice of distribution architecture is a key decision that affects the performance and reliability of the system. This is a major topic in its own right and I cover it separately in Chapter 18.
|
|
||||||
|
|
||||||
The architecture of a software system may be based on a particular architectural pattern or style. An architectural pattern is a description of a system organization (Garlan and Shaw, 1993), such as a client-server organization or a layered architecture. Architectural patterns capture the essence of an architecture that has been used in different software systems. You should be aware of common patterns, where they can be used, and their strengths and weaknesses when making decisions about the architecture of a system. I discuss a number of frequently used patterns in Section 6.3.
|
|
||||||
|
|
||||||
Garlan and Shaw's notion of an architectural style (style and pattern have come to mean the same thing) covers questions 4 to 6 in the previous list. You have to choose the most appropriate structure, such as client-server or layered structuring, that will enable you to meet the system requirements. To decompose structural system units, you decide on the strategy for decomposing components into sub-components. The approaches that you can use allow different types of architecture to be implemented. Finally, in the control modeling process, you make decisions about how the execution of components is controlled. You develop a general model of the control relationships between the various parts of the system.
|
|
||||||
|
|
||||||
Because of the close relationship between non-functional requirements and software architecture, the particular architectural style and structure that you choose for a system should depend on the non-functional system requirements:
|
|
||||||
|
|
||||||
1. Performance If performance is a critical requirement, the architecture should be designed to localize critical operations within a small number of components, with these components all deployed on the same computer rather than distributed across the network. This may mean using a few relatively large components rather than small, fine-grain components, which reduces the number of component communications. You may also consider run-time system organizations that allow the system to be replicated and executed on different processors.
|
|
||||||
2. Security If security is a critical requirement, a layered structure for the architecture should be used, with the most critical assets protected in the innermost layers, with a high level of security validation applied to these layers.
|
|
||||||
3. Safety If safety is a critical requirement, the architecture should be designed so that safety-related operations are all located in either a single component or in a small number of components. This reduces the costs and problems of safety validation and makes it possible to provide related protection systems that can safely shut down the system in the event of failure.
|
|
||||||
4. Availability If availability is a critical requirement, the architecture should be designed to include redundant components so that it is possible to replace and update components without stopping the system. I describe two fault-tolerant system architectures for high-availability systems in Chapter 13.
|
|
||||||
5. Maintainability If maintainability is a critical requirement, the system architecture should be designed using fine-grain, self-contained components that may
|
|
||||||
|
|
||||||
readily be changed. Producers of data should be separated from consumers and shared data structures should be avoided.
|
|
||||||
|
|
||||||
Obviously there is potential conflict between some of these architectures. For example, using large components improves performance and using small, fine-grain components improves maintainability. If both performance and maintainability are important system requirements, then some compromise must be found. This can sometimes be achieved by using different architectural patterns or styles for different parts of the system.
|
|
||||||
|
|
||||||
Evaluating an architectural design is difficult because the true test of an architecture is how well the system meets its functional and non-functional requirements when it is in use. However, you can do some evaluation by comparing your design against reference architectures or generic architectural patterns. Bosch's (2000) description of the non-functional characteristics of architectural patterns can also be used to help with architectural evaluation.
|
|
||||||
|
|
||||||
## 6.2 Architectural views
|
|
||||||
|
|
||||||
I explained in the introduction to this chapter that architectural models of a software system can be used to focus discussion about the software requirements or design. Alternatively, they may be used to document a design so that it can be used as a basis for more detailed design and implementation, and for the future evolution of the system. In this section, I discuss two issues that are relevant to both of these:
|
|
||||||
|
|
||||||
1. What views or perspectives are useful when designing and documenting a system's architecture?
|
|
||||||
2. What notations should be used for describing architectural models?
|
|
||||||
|
|
||||||
It is impossible to represent all relevant information about a system's architecture in a single architectural model, as each model only shows one view or perspective of the system. It might show how a system is decomposed into modules, how the run-time processes interact, or the different ways in which system components are distributed across a network. All of these are useful at different times so, for both design and documentation, you usually need to present multiple views of the software architecture.
|
|
||||||
|
|
||||||
There are different opinions as to what views are required. Krutchen (1995), in his well-known 4+1 view model of software architecture, suggests that there should be four fundamental architectural views, which are related using use cases or scenarios. The views that he suggests are:
|
|
||||||
|
|
||||||
1. A logical view, which shows the key abstractions in the system as objects or object classes. It should be possible to relate the system requirements to entities in this logical view.
|
|
||||||
|
|
||||||
2. A process view, which shows how, at run-time, the system is composed of interacting processes. This view is useful for making judgments about nonfunctional system characteristics such as performance and availability.
|
|
||||||
3. A development view, which shows how the software is decomposed for development, that is, it shows the breakdown of the software into components that are implemented by a single developer or development team. This view is useful for software managers and programmers.
|
|
||||||
4. A physical view, which shows the system hardware and how software components are distributed across the processors in the system. This view is useful for systems engineers planning a system deployment.
|
|
||||||
|
|
||||||
Hofmeister et al. (2000) suggest the use of similar views but add to this the notion of a conceptual view. This view is an abstract view of the system that can be the basis for decomposing high-level requirements into more detailed specifications, help engineers make decisions about components that can be reused, and represent a product line (discussed in Chapter 16) rather than a single system. Figure 6.1, which describes the architecture of a packing robot, is an example of a conceptual system view.
|
|
||||||
|
|
||||||
In practice, conceptual views are almost always developed during the design process and are used to support architectural decision making. They are a way of communicating the essence of a system to different stakeholders. During the design process, some of the other views may also be developed when different aspects of the system are discussed, but there is no need for a complete description from all perspectives. It may also be possible to associate architectural patterns, discussed in the next section, with the different views of a system.
|
|
||||||
|
|
||||||
There are differing views about whether or not software architects should use the UML for architectural description (Clements, et al., 2002). A survey in 2006 (Lange et al., 2006) showed that, when the UML was used, it was mostly applied in a loose and informal way. The authors of that paper argued that this was a bad thing. I disagree with this view. The UML was designed for describing object-oriented systems and, at the architectural design stage, you often want to describe systems at a higher level of abstraction. Object classes are too close to the implementation to be useful for architectural description.
|
|
||||||
|
|
||||||
I don't find the UML to be useful during the design process itself and prefer informal notations that are quicker to write and which can be easily drawn on a whiteboard. The UML is of most value when you are documenting an architecture in detail or using model-driven development, as discussed in Chapter 5.
|
|
||||||
|
|
||||||
A number of researchers have proposed the use of more specialized architectural description languages (ADLs) (Bass et al., 2003) to describe system architectures. The basic elements of ADLs are components and connectors, and they include rules and guidelines for well-formed architectures. However, because of their specialized nature, domain and application specialists find it hard to understand and use ADLs. This makes it difficult to assess their usefulness for practical software engineering. ADLs designed for a particular domain (e.g., automobile systems) may be used as a
|
|
||||||
|
|
||||||
■
|
|
||||||
|
|
||||||
| Name | MVC (Model-View-Controller) |
|
|
||||||
|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
||||||
| Description | Separates presentation and interaction from the system data. The system is structured into three logical components that interact with each other. The Model component manages the system data and associated operations on that data. The View component defines and manages how the data is presented to the user. The Controller component manages user interaction (e.g., key presses, mouse clicks, etc.) and passes these interactions to the View and the Model. See Figure 6.3. |
|
|
||||||
| Example | Figure 6.4 shows the architecture of a web-based application system organized using the MVC pattern. |
|
|
||||||
| When used | Used when there are multiple ways to view and interact with data. Also used when the future requirements for interaction and presentation of data are unknown. |
|
|
||||||
| Advantages | Allows the data to change independently of its representation and vice versa. Supports presentation of the same data in different ways with changes made in one representation shown in all of them. |
|
|
||||||
| Disadvantages | Can involve additional code and code complexity when the data model and interactions are simple. |
|
|
||||||
|
|
||||||
Figure 6.2 The modelview-controller (MVC) pattern basis for model-driven development. However, I believe that informal models and notations, such as the UML, will remain the most commonly used ways of documenting system architectures.
|
|
||||||
|
|
||||||
Users of agile methods claim that detailed design documentation is mostly unused. It is, therefore, a waste of time and money to develop it. I largely agree with this view and I think that, for most systems, it is not worth developing a detailed architectural description from these four perspectives. You should develop the views that are useful for communication and not worry about whether or not your architectural documentation is complete. However, an exception to this is when you are developing critical systems, when you need to make a detailed dependability analysis of the system. You may need to convince external regulators that your system conforms to their regulations and so complete architectural documentation may be required.
|
|
||||||
|
|
||||||
## 6.3 Architectural patterns
|
|
||||||
|
|
||||||
The idea of patterns as a way of presenting, sharing, and reusing knowledge about software systems is now widely used. The trigger for this was the publication of a book on object-oriented design patterns (Gamma et al., 1995), which prompted the development of other types of pattern, such as patterns for organizational design (Coplien and Harrison, 2004), usability patterns (Usability Group, 1998), interaction (Martin and Sommerville, 2004), configuration management (Berczuk and Appleton, 2002), and so on. Architectural patterns were proposed in the 1990s under the name 'architectural styles' (Shaw and Garlan, 1996), with a five-volume series of handbooks on pattern-oriented software architecture published between 1996 and 2007 (Buschmann et al., 1996; Buschmann et al., 2007a; Buschmann et al., 2007b; Kircher and Jain, 2004; Schmidt et al., 2000).
|
|
||||||
|
|
||||||
Figure 6.3 The organization of the MVC
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
In this section, I introduce architectural patterns and briefly describe a selection of architectural patterns that are commonly used in different types of systems. For more information about patterns and their use, you should refer to published pattern handbooks.
|
|
||||||
|
|
||||||
You can think of an architectural pattern as a stylized, abstract description of good practice, which has been tried and tested in different systems and environments. So, an architectural pattern should describe a system organization that has been successful in previous systems. It should include information of when it is and is not appropriate to use that pattern, and the pattern's strengths and weaknesses.
|
|
||||||
|
|
||||||
For example, Figure 6.2 describes the well-known Model-View-Controller pattern. This pattern is the basis of interaction management in many web-based systems. The stylized pattern description includes the pattern name, a brief description (with an associated graphical model), and an example of the type of system where the pattern is used (again, perhaps with a graphical model). You should also include information about when the pattern should be used and its advantages and disadvantages. Graphical models of the architecture associated with the MVC pattern are shown in Figures 6.3 and 6.4. These present the architecture from different views-Figure 6.3 is a conceptual view and Figure 6.4 shows a possible run-time architecture when this pattern is used for interaction management in a web-based system.
|
|
||||||
|
|
||||||
In a short section of a general chapter, it is impossible to describe all of the generic patterns that can be used in software development. Rather, I present some selected examples of patterns that are widely used and which capture good architectural design principles. I have included some further examples of generic architectural patterns on the book's web pages.
|
|
||||||
|
|
||||||
Figure 6.4 Web application architecture using the MVC pattern
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
## 6.3.1 Layered architecture
|
|
||||||
|
|
||||||
The notions of separation and independence are fundamental to architectural design because they allow changes to be localized. The MVC pattern, shown in Figure 6.2, separates elements of a system, allowing them to change independently. For example, adding a new view or changing an existing view can be done without any changes to the underlying data in the model. The layered architecture pattern is another way of achieving separation and independence. This pattern is shown in Figure 6.5. Here, the system functionality is organized into separate layers, and each layer only relies on the facilities and services offered by the layer immediately beneath it.
|
|
||||||
|
|
||||||
This layered approach supports the incremental development of systems. As a layer is developed, some of the services provided by that layer may be made available to users. The architecture is also changeable and portable. So long as its interface is unchanged, a layer can be replaced by another, equivalent layer. Furthermore, when layer interfaces change or new facilities are added to a layer, only the adjacent layer is affected. As layered systems localize machine dependencies in inner layers, this makes it easier to provide multi-platform implementations of an application system. Only the inner, machine-dependent layers need be re-implemented to take account of the facilities of a different operating system or database.
|
|
||||||
|
|
||||||
Figure 6.6 is an example of a layered architecture with four layers. The lowest layer includes system support software-typically database and operating system support. The next layer is the application layer that includes the components concerned with the application functionality and utility components that are used by other application components. The third layer is concerned with user interface
|
|
||||||
|
|
||||||
| Name | Layered architecture |
|
|
||||||
|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
||||||
| Description | Organizes the system into layers with related functionality associated with each layer. A layer provides services to the layer above it so the lowest-level layers represent core services that are likely to be used throughout the system. See Figure 6.6. |
|
|
||||||
| Example | A layered model of a system for sharing copyright documents held in different libraries, as shown in Figure 6.7. |
|
|
||||||
| When used | Used when building new facilities on top of existing systems; when the development is spread across several teams with each team responsibility for a layer of functionality; when there is a requirement for multi-level security. |
|
|
||||||
| Advantages | Allows replacement of entire layers so long as the interface is maintained. Redundant facilities (e.g., authentication) can be provided in each layer to increase the dependability of the system. |
|
|
||||||
| Disadvantages | In practice, providing a clean separation between layers is often difficult and a high-level layer may have to interact directly with lower-level layers rather than through the layer immediately below it. Performance can be a problem because of multiple levels of interpretation of a service request as it is processed at each layer. |
|
|
||||||
|
|
||||||
## Figure 6.5 The layered architecture pattern
|
|
||||||
|
|
||||||
management and providing user authentication and authorization, with the top layer providing user interface facilities. Of course, the number of layers is arbitrary. Any of the layers in Figure 6.6 could be split into two or more layers.
|
|
||||||
|
|
||||||
Figure 6.7 is an example of how this layered architecture pattern can be applied to a library system called LIBSYS, which allows controlled electronic access to copyright material from a group of university libraries. This has a five-layer architecture, with the bottom layer being the individual databases in each library.
|
|
||||||
|
|
||||||
You can see another example of the layered architecture pattern in Figure 6.17 (found in Section 6.4). This shows the organization of the system for mental healthcare (MHC-PMS) that I have discussed in earlier chapters.
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
Figure 6.7 The architecture of the LIBSYS system
|
|
||||||
|
|
||||||
■
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
## 6.3.2 Repository architecture
|
|
||||||
|
|
||||||
The layered architecture and MVC patterns are examples of patterns where the view presented is the conceptual organization of a system. My next example, the Repository pattern (Figure 6.8), describes how a set of interacting components can share data.
|
|
||||||
|
|
||||||
Figure 6.8 The repository pattern The majority of systems that use large amounts of data are organized around a shared database or repository. This model is therefore suited to applications in which data is generated by one component and used by another. Examples of this type of system include command and control systems, management information systems, CAD systems, and interactive development environments for software.
|
|
||||||
|
|
||||||
| Name | Repository |
|
|
||||||
|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
||||||
| Description | All data in a system is managed in a central repository that is accessible to all system components. Components do not interact directly, only through the repository. |
|
|
||||||
| Example | Figure 6.9 is an example of an IDE where the components use a repository of system design information. Each software tool generates information which is then available for use by other tools. |
|
|
||||||
| When used | You should use this pattern when you have a system in which large volumes of information are generated that has to be stored for a long time. You may also use it in data-driven systems where the inclusion of data in the repository triggers an action or tool. |
|
|
||||||
| Advantages | Components can be independent-they do not need to know of the existence of other components. Changes made by one component can be propagated to all components. All data can be managed consistently (e.g., backups done at the same time) as it is all in one place. |
|
|
||||||
| Disadvantages | The repository is a single point of failure so problems in the repository affect the whole system. May be inefficiencies in organizing all communication through the repository. Distributing the repository across several computers may be difficult. |
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
Figure 6.9 is an illustration of a situation in which a repository might be used. This diagram shows an IDE that includes different tools to support model-driven development. The repository in this case might be a version-controlled environment (as discussed in Chapter 25) that keeps track of changes to software and allows rollback to earlier versions.
|
|
||||||
|
|
||||||
Organizing tools around a repository is an efficient way to share large amounts of data. There is no need to transmit data explicitly from one component to another. However, components must operate around an agreed repository data model. Inevitably, this is a compromise between the specific needs of each tool and it may be difficult or impossible to integrate new components if their data models do not fit the agreed schema. In practice, it may be difficult to distribute the repository over a number of machines. Although it is possible to distribute a logically centralized repository, there may be problems with data redundancy and inconsistency.
|
|
||||||
|
|
||||||
In the example shown in Figure 6.9, the repository is passive and control is the responsibility of the components using the repository. An alternative approach, which has been derived for AI systems, uses a 'blackboard' model that triggers components when particular data become available. This is appropriate when the form of the repository data is less well structured. Decisions about which tool to activate can only be made when the data has been analyzed. This model is introduced by Nii (1986). Bosch (2000) includes a good discussion of how this style relates to system quality attributes.
|
|
||||||
|
|
||||||
## 6.3.3 Client-server architecture
|
|
||||||
|
|
||||||
The repository pattern is concerned with the static structure of a system and does not show its run-time organization. My next example illustrates a very commonly used run-time organization for distributed systems. The Client-server pattern is described in Figure 6.10.
|
|
||||||
|
|
||||||
■
|
|
||||||
|
|
||||||
| Name | Client-server |
|
|
||||||
|---------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
||||||
| Description | In a client-server architecture, the functionality of the system is organized into services, with each service delivered from a separate server. Clients are users of these services and access servers to make use of them. |
|
|
||||||
| Example | Figure 6.11 is an example of a film and video/DVD library organized as a client-server system. |
|
|
||||||
| When used | Used when data in a shared database has to be accessed from a range of locations. Because servers can be replicated, may also be used when the load on a system is variable. |
|
|
||||||
| Advantages | The principal advantage of this model is that servers can be distributed across a network. General functionality (e.g., a printing service) can be available to all clients and does not need to be implemented by all services. |
|
|
||||||
| Disadvantages | Each service is a single point of failure so susceptible to denial of service attacks or server failure. Performance may be unpredictable because it depends on the network as well as the system. May be management problems if servers are owned by different organizations. |
|
|
||||||
|
|
||||||
Figure 6.10 The client-server pattern A system that follows the client-server pattern is organized as a set of services and associated servers, and clients that access and use the services. The major components of this model are:
|
|
||||||
|
|
||||||
1. A set of servers that offer services to other components. Examples of servers include print servers that offer printing services, file servers that offer file management services, and a compile server, which offers programming language compilation services.
|
|
||||||
2. A set of clients that call on the services offered by servers. There will normally be several instances of a client program executing concurrently on different computers.
|
|
||||||
3. A network that allows the clients to access these services. Most client-server systems are implemented as distributed systems, connected using Internet protocols.
|
|
||||||
|
|
||||||
Client-server architectures are usually thought of as distributed systems architectures but the logical model of independent services running on separate servers can be implemented on a single computer. Again, an important benefit is separation and independence. Services and servers can be changed without affecting other parts of the system.
|
|
||||||
|
|
||||||
Clients may have to know the names of the available servers and the services that they provide. However, servers do not need to know the identity of clients or how many clients are accessing their services. Clients access the services provided by a server through remote procedure calls using a request-reply protocol such as the http protocol used in the WWW. Essentially, a client makes a request to a server and waits until it receives a reply.
|
|
||||||
|
|
||||||
Figure 6.11 A clientserver architecture for a film library
|
|
||||||
|
|
||||||
Figure 6.12 The pipe and filter pattern
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
Figure 6.11 is an example of a system that is based on the client-server model. This is a multi-user, web-based system for providing a film and photograph library. In this system, several servers manage and display the different types of media. Video frames need to be transmitted quickly and in synchrony but at relatively low resolution. They may be compressed in a store, so the video server can handle video compression and decompression in different formats. Still pictures, however, must be maintained at a high resolution, so it is appropriate to maintain them on a separate server.
|
|
||||||
|
|
||||||
The catalog must be able to deal with a variety of queries and provide links into the web information system that includes data about the film and video clips, and an e-commerce system that supports the sale of photographs, film, and video clips. The
|
|
||||||
|
|
||||||
| Name | Pipe and filter |
|
|
||||||
|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
||||||
| Description | The processing of the data in a system is organized so that each processing component (filter) is discrete and carries out one type of data transformation. The data flows (as in a pipe) from one component to another for processing. |
|
|
||||||
| Example | Figure 6.13 is an example of a pipe and filter system used for processing invoices. |
|
|
||||||
| When used | Commonly used in data processing applications (both batch- and transaction-based) where inputs are processed in separate stages to generate related outputs. |
|
|
||||||
| Advantages | Easy to understand and supports transformation reuse. Workflow style matches the structure of many business processes. Evolution by adding transformations is straightforward. Can be implemented as either a sequential or concurrent system. |
|
|
||||||
| Disadvantages | The format for data transfer has to be agreed upon between communicating transformations. Each transformation must parse its input and unparse its output to the agreed form. This increases system overhead and may mean that it is impossible to reuse functional transformations that use incompatible data structures. |
|
|
||||||
|
|
||||||
■
|
|
||||||
|
|
||||||
Figure 6.13 An example of the pipe and filter architecture
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
client program is simply an integrated user interface, constructed using a web browser, to access these services.
|
|
||||||
|
|
||||||
The most important advantage of the client-server model is that it is a distributed architecture. Effective use can be made of networked systems with many distributed processors. It is easy to add a new server and integrate it with the rest of the system or to upgrade servers transparently without affecting other parts of the system. I discuss distributed architectures, including client-server architectures and distributed object architectures, in Chapter 18.
|
|
||||||
|
|
||||||
## 6.3.4 Pipe and filter architecture
|
|
||||||
|
|
||||||
My final example of an architectural pattern is the pipe and filter pattern. This is a model of the run-time organization of a system where functional transformations process their inputs and produce outputs. Data flows from one to another and is transformed as it moves through the sequence. Each processing step is implemented as a transform. Input data flows through these transforms until converted to output. The transformations may execute sequentially or in parallel. The data can be processed by each transform item by item or in a single batch.
|
|
||||||
|
|
||||||
The name 'pipe and filter' comes from the original Unix system where it was possible to link processes using 'pipes'. These passed a text stream from one process to another. Systems that conform to this model can be implemented by combining Unix commands, using pipes and the control facilities of the Unix shell. The term 'filter' is used because a transformation 'filters out' the data it can process from its input data stream.
|
|
||||||
|
|
||||||
Variants of this pattern have been in use since computers were first used for automatic data processing. When transformations are sequential with data processed in batches, this pipe and filter architectural model becomes a batch sequential model, a common architecture for data processing systems (e.g., a billing system). The architecture of an embedded system may also be organized as a process pipeline, with each process executing concurrently. I discuss the use of this pattern in embedded systems in Chapter 20.
|
|
||||||
|
|
||||||
An example of this type of system architecture, used in a batch processing application, is shown in Figure 6.13. An organization has issued invoices to customers. Once a week, payments that have been made are reconciled with the invoices. For
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
## Architectural patterns for control
|
|
||||||
|
|
||||||
There are specific architectural patterns that reflect commonly used ways of organizing control in a system. These include centralized control, based on one component calling other components, and event-based control, where the system reacts to external events.
|
|
||||||
|
|
||||||
http://www.SoftwareEngineering-9.com/Web/Architecture/ArchPatterns/
|
|
||||||
|
|
||||||
those invoices that have been paid, a receipt is issued. For those invoices that have not been paid within the allowed payment time, a reminder is issued.
|
|
||||||
|
|
||||||
Interactive systems are difficult to write using the pipe and filter model because of the need for a stream of data to be processed. Although simple textual input and output can be modeled in this way, graphical user interfaces have more complex I/O formats and a control strategy that is based on events such as mouse clicks or menu selections. It is difficult to translate this into a form compatible with the pipelining model.
|
|
||||||
|
|
||||||
## 6.4 Application architectures
|
|
||||||
|
|
||||||
Application systems are intended to meet a business or organizational need. All businesses have much in common-they need to hire people, issue invoices, keep accounts, and so on. Businesses operating in the same sector use common sectorspecific applications. Therefore, as well as general business functions, all phone companies need systems to connect calls, manage their network, issue bills to customers, etc. Consequently, the application systems used by these businesses also have much in common.
|
|
||||||
|
|
||||||
These commonalities have led to the development of software architectures that describe the structure and organization of particular types of software systems. Application architectures encapsulate the principal characteristics of a class of systems. For example, in real-time systems, there might be generic architectural models of different system types, such as data collection systems or monitoring systems. Although instances of these systems differ in detail, the common architectural structure can be reused when developing new systems of the same type.
|
|
||||||
|
|
||||||
The application architecture may be re-implemented when developing new systems but, for many business systems, application reuse is possible without reimplementation. We see this in the growth of Enterprise Resource Planning (ERP) systems from companies such as SAP and Oracle, and vertical software packages (COTS) for specialized applications in different areas of business. In these systems, a generic system is configured and adapted to create a specific business application.
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
## Application architectures
|
|
||||||
|
|
||||||
There are several examples of application architectures on the book's website. These include descriptions of batch data-processing systems, resource allocation systems, and event-based editing systems.
|
|
||||||
|
|
||||||
http://www.SoftwareEngineering-9.com/Web/Architecture/AppArch/
|
|
||||||
|
|
||||||
For example, a system for supply chain management can be adapted for different types of suppliers, goods, and contractual arrangements.
|
|
||||||
|
|
||||||
As a software designer, you can use models of application architectures in a number of ways:
|
|
||||||
|
|
||||||
1. As a starting point for the architectural design process If you are unfamiliar with the type of application that you are developing, you can base your initial design on a generic application architecture. Of course, this will have to be specialized for the specific system being developed, but it is a good starting point for design.
|
|
||||||
2. As a design checklist If you have developed an architectural design for an application system, you can compare this with the generic application architecture. You can check that your design is consistent with the generic architecture.
|
|
||||||
3. As a way of organizing the work of the development team The application architectures identify stable structural features of the system architectures and in many cases, it is possible to develop these in parallel. You can assign work to group members to implement different components within the architecture.
|
|
||||||
4. As a means of assessing components for reuse If you have components you might be able to reuse, you can compare these with the generic structures to see whether there are comparable components in the application architecture.
|
|
||||||
5. As a vocabulary for talking about types of applications If you are discussing a specific application or trying to compare applications of the same types, then you can use the concepts identified in the generic architecture to talk about the applications.
|
|
||||||
|
|
||||||
There are many types of application system and, in some cases, they may seem to be very different. However, many of these superficially dissimilar applications actually have much in common, and thus can be represented by a single abstract application architecture. I illustrate this here by describing the following architectures of two types of application:
|
|
||||||
|
|
||||||
1. Transaction processing applications Transaction processing applications are database-centered applications that process user requests for information and update the information in a database. These are the most common type of interactive business systems. They are organized in such a way that user actions can't interfere with each other and the integrity of the database is maintained. This
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
- class of system includes interactive banking systems, e-commerce systems, information systems, and booking systems.
|
|
||||||
2. Language processing systems Language processing systems are systems in which the user's intentions are expressed in a formal language (such as Java). The language processing system processes this language into an internal format and then interprets this internal representation. The best-known language processing systems are compilers, which translate high-level language programs into machine code. However, language processing systems are also used to interpret command languages for databases and information systems, and markup languages such as XML (Harold and Means, 2002; Hunter et al., 2007).
|
|
||||||
|
|
||||||
I have chosen these particular types of system because a large number of webbased business systems are transaction-processing systems, and all software development relies on language processing systems.
|
|
||||||
|
|
||||||
## 6.4.1 Transaction processing systems
|
|
||||||
|
|
||||||
Transaction processing (TP) systems are designed to process user requests for information from a database, or requests to update a database (Lewis et al., 2003). Technically, a database transaction is sequence of operations that is treated as a single unit (an atomic unit). All of the operations in a transaction have to be completed before the database changes are made permanent. This ensures that failure of operations within the transaction does not lead to inconsistencies in the database.
|
|
||||||
|
|
||||||
From a user perspective, a transaction is any coherent sequence of operations that satisfies a goal, such as 'find the times of flights from London to Paris'. If the user transaction does not require the database to be changed then it may not be necessary to package this as a technical database transaction.
|
|
||||||
|
|
||||||
An example of a transaction is a customer request to withdraw money from a bank account using an ATM. This involves getting details of the customer's account, checking the balance, modifying the balance by the amount withdrawn, and sending commands to the ATM to deliver the cash. Until all of these steps have been completed, the transaction is incomplete and the customer accounts database is not changed.
|
|
||||||
|
|
||||||
Transaction processing systems are usually interactive systems in which users make asynchronous requests for service. Figure 6.14 illustrates the conceptual architectural structure of TP applications. First a user makes a request to the system through an I/O processing component. The request is processed by some applicationspecific logic. A transaction is created and passed to a transaction manager, which is usually embedded in the database management system. After the transaction manager has ensured that the transaction is properly completed, it signals to the application that processing has finished.
|
|
||||||
|
|
||||||
Figure 6.15 The software architecture of an ATM system
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
Transaction processing systems may be organized as a 'pipe and filter' architecture with system components responsible for input, processing, and output. For example, consider a banking system that allows customers to query their accounts and withdraw cash from an ATM. The system is composed of two cooperating software components-the ATM software and the account processing software in the bank's database server. The input and output components are implemented as software in the ATM and the processing component is part of the bank's database server. Figure 6.15 shows the architecture of this system, illustrating the functions of the input, process, and output components.
|
|
||||||
|
|
||||||
## 6.4.2 Information systems
|
|
||||||
|
|
||||||
All systems that involve interaction with a shared database can be considered to be transaction-based information systems. An information system allows controlled access to a large base of information, such as a library catalog, a flight timetable, or the records of patients in a hospital. Increasingly, information systems are web-based systems that are accessed through a web browser.
|
|
||||||
|
|
||||||
Figure 6.16 a very general model of an information system. The system is modeled using a layered approach (discussed in Section 6.3) where the top layer supports the user interface and the bottom layer is the system database. The user communications layer handles all input and output from the user interface, and the information retrieval layer includes application-specific logic for accessing and updating the database. As we shall see later, the layers in this model can map directly onto servers in an Internet-based system.
|
|
||||||
|
|
||||||
As an example of an instantiation of this layered model, Figure 6.17 shows the architecture of the MHC-PMS. Recall that this system maintains and manages details of patients who are consulting specialist doctors about mental health problems. I have Figure 6.16 Layered information system architecture added detail to each layer in the model by identifying the components that support user communications and information retrieval and access:
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
1. The top layer is responsible for implementing the user interface. In this case, the UI has been implemented using a web browser.
|
|
||||||
2. The second layer provides the user interface functionality that is delivered through the web browser. It includes components to allow users to log in to the system and checking components that ensure that the operations they use are allowed by their role. This layer includes form and menu management components that present information to users, and data validation components that check information consistency.
|
|
||||||
3. The third layer implements the functionality of the system and provides components that implement system security, patient information creation and updating, import and export of patient data from other databases, and report generators that create management reports.
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
■
|
|
||||||
|
|
||||||
4. Finally, the lowest layer, which is built using a commercial database management system, provides transaction management and persistent data storage.
|
|
||||||
|
|
||||||
Information and resource management systems are now usually web-based systems where the user interfaces are implemented using a web browser. For example, e-commerce systems are Internet-based resource management systems that accept electronic orders for goods or services and then arrange delivery of these goods or services to the customer. In an e-commerce system, the application-specific layer includes additional functionality supporting a 'shopping cart' in which users can place a number of items in separate transactions, then pay for them all together in a single transaction.
|
|
||||||
|
|
||||||
The organization of servers in these systems usually reflects the four-layer generic model presented in Figure 6.16. These systems are often implemented as multi-tier client server/architectures, as discussed in Chapter 18:
|
|
||||||
|
|
||||||
1. The web server is responsible for all user communications, with the user interface implemented using a web browser;
|
|
||||||
2. The application server is responsible for implementing application-specific logic as well as information storage and retrieval requests;
|
|
||||||
3. The database server moves information to and from the database and handles transaction management.
|
|
||||||
|
|
||||||
Using multiple servers allows high throughput and makes it possible to handle hundreds of transactions per minute. As demand increases, servers can be added at each level to cope with the extra processing involved.
|
|
||||||
|
|
||||||
## 6.4.3 Language processing systems
|
|
||||||
|
|
||||||
Language processing systems translate a natural or artificial language into another representation of that language and, for programming languages, may also execute the resulting code. In software engineering, compilers translate an artificial programming language into machine code. Other language-processing systems may translate an XML data description into commands to query a database or to an alternative XML representation. Natural language processing systems may translate one natural language to another e.g., French to Norwegian.
|
|
||||||
|
|
||||||
A possible architecture for a language processing system for a programming language is illustrated in Figure 6.18. The source language instructions define the program to be executed and a translator converts these into instructions for an abstract machine. These instructions are then interpreted by another component that fetches the instructions for execution and executes them using (if necessary) data from the environment. The output of the process is the result of interpreting the instructions on the input data.
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
Of course, for many compilers, the interpreter is a hardware unit that processes machine instructions and the abstract machine is a real processor. However, for dynamically typed languages, such as Python, the interpreter may be a software component.
|
|
||||||
|
|
||||||
Programming language compilers that are part of a more general programming environment have a generic architecture (Figure 6.19) that includes the following components:
|
|
||||||
|
|
||||||
1. A lexical analyzer, which takes input language tokens and converts them to an internal form.
|
|
||||||
2. A symbol table, which holds information about the names of entities (variables, class names, object names, etc.) used in the text that is being translated.
|
|
||||||
3. A syntax analyzer, which checks the syntax of the language being translated. It uses a defined grammar of the language and builds a syntax tree.
|
|
||||||
4. A syntax tree, which is an internal structure representing the program being compiled.
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
Figure 6.19 A pipe and filter compiler architecture
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
## Reference architectures
|
|
||||||
|
|
||||||
Reference architectures capture important features of system architectures in a domain. Essentially, they include everything that might be in an application architecture although, in reality, it is very unlikely that any individual application would include all the features shown in a reference architecture. The main purpose of reference architectures is to evaluate and compare design proposals, and to educate people about architectural characteristics in that domain.
|
|
||||||
|
|
||||||
http://www.SoftwareEngineering-9.com/Web/Architecture/RefArch.html
|
|
||||||
|
|
||||||
5. A semantic analyzer that uses information from the syntax tree and the symbol table to check the semantic correctness of the input language text.
|
|
||||||
6. A code generator that 'walks' the syntax tree and generates abstract machine code.
|
|
||||||
|
|
||||||
Other components might also be included which analyze and transform the syntax tree to improve efficiency and remove redundancy from the generated machine code. In other types of language processing system, such as a natural language translator, there will be additional components such as a dictionary, and the generated code is actually the input text translated into another language.
|
|
||||||
|
|
||||||
There are alternative architectural patterns that may be used in a language processing system (Garlan and Shaw, 1993). Compilers can be implemented using a composite of a repository and a pipe and filter model. In a compiler architecture, the symbol table is a repository for shared data. The phases of lexical, syntactic, and semantic analysis are organized sequentially, as shown in Figure 6.19, and communicate through the shared symbol table.
|
|
||||||
|
|
||||||
This pipe and filter model of language compilation is effective in batch environments where programs are compiled and executed without user interaction; for example, in the translation of one XML document to another. It is less effective when a compiler is integrated with other language processing tools such as a structured editing system, an interactive debugger or a program prettyprinter. In this situation, changes from one component need to be reflected immediately in other components. It is better, therefore, to organize the system around a repository, as shown in Figure 6.20.
|
|
||||||
|
|
||||||
This figure illustrates how a language processing system can be part of an integrated set of programming support tools. In this example, the symbol table and syntax tree act as a central information repository. Tools or tool fragments communicate through it. Other information that is sometimes embedded in tools, such as the grammar definition and the definition of the output format for the program, have been taken out of the tools and put into the repository. Therefore, a syntax-directed editor can check that the syntax of a program is correct as it is being typed and a prettyprinter can create listings of the program in a format that is easy to read.
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
## KEY POINTS
|
|
||||||
|
|
||||||
- ■ A software architecture is a description of how a software system is organized. Properties of a system such as performance, security, and availability are influenced by the architecture used.
|
|
||||||
- ■ Architectural design decisions include decisions on the type of application, the distribution of the system, the architectural styles to be used, and the ways in which the architecture should be documented and evaluated.
|
|
||||||
- ■ Architectures may be documented from several different perspectives or views. Possible views include a conceptual view, a logical view, a process view, a development view, and a physical view.
|
|
||||||
- ■ Architectural patterns are a means of reusing knowledge about generic system architectures. They describe the architecture, explain when it may be used, and discuss its advantages and disadvantages.
|
|
||||||
- ■ Commonly used architectural patterns include Model-View-Controller, Layered Architecture, Repository, Client-server, and Pipe and Filter.
|
|
||||||
- ■ Generic models of application systems architectures help us understand the operation of applications, compare applications of the same type, validate application system designs, and assess large-scale components for reuse.
|
|
||||||
- ■ Transaction processing systems are interactive systems that allow information in a database to be remotely accessed and modified by a number of users. Information systems and resource management systems are examples of transaction processing systems.
|
|
||||||
- ■ Language processing systems are used to translate texts from one language into another and to carry out the instructions specified in the input language. They include a translator and an abstract machine that executes the generated language.
|
|
||||||
|
|
||||||
## FURTHER READING
|
|
||||||
|
|
||||||
Software Architecture: Perspectives on an Emerging Discipline. This was the first book on software architecture and has a good discussion on different architectural styles. (M. Shaw and D. Garlan, Prentice-Hall, 1996.)
|
|
||||||
|
|
||||||
Software Architecture in Practice, 2nd ed. This is a practical discussion of software architectures that does not oversell the benefits of architectural design. It provides a clear business rationale explaining why architectures are important. (L. Bass, P. Clements and R. Kazman, Addison-Wesley, 2003.)
|
|
||||||
|
|
||||||
'The Golden Age of Software Architecture' This paper surveys the development of software architecture from its beginnings in the 1980s through to its current usage. There is little technical content but it is an interesting historical overview. (M. Shaw and P. Clements, IEEE Software, 21 (2), March-April 2006.) http://dx.doi.org/10.1109/MS.2006.58.
|
|
||||||
|
|
||||||
Handbook of Software Architecture . This is a work in progress by Grady Booch, one of the early evangelists for software architecture. He has been documenting the architectures of a range of software systems so you can see reality rather than academic abstraction. Available on the Web and intended to appear as a book. http://www.handbookofsoftwarearchitecture.com/.
|
|
||||||
|
|
||||||
## EXERCISES
|
|
||||||
|
|
||||||
- 6.1. When describing a system, explain why you may have to design the system architecture before the requirements specification is complete.
|
|
||||||
- 6.2. You have been asked to prepare and deliver a presentation to a non-technical manager to justify the hiring of a system architect for a new project. Write a list of bullet points setting out the key points in your presentation. Naturally, you have to explain what is meant by system architecture.
|
|
||||||
- 6.3. Explain why design conflicts might arise when designing an architecture for which both availability and security requirements are the most important non-functional requirements.
|
|
||||||
- 6.4. Draw diagrams showing a conceptual view and a process view of the architectures of the following systems:
|
|
||||||
|
|
||||||
An automated ticket-issuing system used by passengers at a railway station.
|
|
||||||
|
|
||||||
A computer-controlled video conferencing system that allows video, audio, and computer data to be visible to several participants at the same time.
|
|
||||||
|
|
||||||
A robot floor cleaner that is intended to clean relatively clear spaces such as corridors. The cleaner must be able to sense walls and other obstructions.
|
|
||||||
|
|
||||||
<!-- image -->
|
|
||||||
|
|
||||||
- 6.5. Explain why you normally use several architectural patterns when designing the architecture of a large system. Apart from the information about patterns that I have discussed in this chapter, what additional information might be useful when designing large systems?
|
|
||||||
- 6.6. Suggest an architecture for a system (such as iTunes) that is used to sell and distribute music on the Internet. What architectural patterns are the basis for this architecture?
|
|
||||||
- 6.7. Explain how you would use the reference model of CASE environments (available on the book's web pages) to compare the IDEs offered by different vendors of a programming language such as Java.
|
|
||||||
- 6.8. Using the generic model of a language processing system presented here, design the architecture of a system that accepts natural language commands and translates these into database queries in a language such as SQL.
|
|
||||||
- 6.9. Using the basic model of an information system, as presented in Figure 6.16, suggest the components that might be part of an information system that allows users to view information about flights arriving and departing from a particular airport.
|
|
||||||
- 6.10. Should there be a separate profession of 'software architect' whose role is to work independently with a customer to design the software system architecture? A separate software company would then implement the system. What might be the difficulties of establishing such a profession?
|
|
||||||
|
|
||||||
## REFERENCES
|
|
||||||
|
|
||||||
Bass, L., Clements, P. and Kazman, R. (2003). Software Architecture in Practice, 2nd ed. Boston: Addison-Wesley.
|
|
||||||
|
|
||||||
Berczuk, S. P. and Appleton, B. (2002). Software Configuration Management Patterns: Effective Teamwork, Practical Integration. Boston: Addison-Wesley.
|
|
||||||
|
|
||||||
Booch, G. (2009). 'Handbook of software architecture'. Web publication. http:/ /www.handbookofsoftwarearchitecture.com/.
|
|
||||||
|
|
||||||
Bosch, J. (2000). Design and Use of Software Architectures. Harlow, UK: Addison-Wesley.
|
|
||||||
|
|
||||||
Buschmann, F., Henney, K. and Schmidt, D. C. (2007a). Pattern-oriented Software Architecture Volume 4: A Pattern Language for Distributed Computing. New York: John Wiley & Sons.
|
|
||||||
|
|
||||||
Buschmann, F., Henney, K. and Schmidt, D. C. (2007b). Pattern-oriented Software Architecture Volume 5: On Patterns and Pattern Languages. New York: John Wiley & Sons.
|
|
||||||
|
|
||||||
Buschmann, F., Meunier, R., Rohnert, H. and Sommerlad, P. (1996). Pattern-oriented Software Architecture Volume 1: A System of Patterns. New York: John Wiley & Sons.
|
|
||||||
|
|
||||||
■
|
|
||||||
|
|
||||||
Clements, P., Bachmann, F., Bass, L., Garlan, D., Ivers, J., Little, R., Nord, R. and Stafford, J. (2002). Documenting Software Architectures: Views and Beyond. Boston: Addison-Wesley.
|
|
||||||
|
|
||||||
Coplien, J. H. and Harrison, N. B. (2004). Organizational Patterns of Agile Software Development. Englewood Cliffs, NJ: Prentice Hall.
|
|
||||||
|
|
||||||
Gamma, E., Helm, R., Johnson, R. and Vlissides, J. (1995). Design Patterns: Elements of Reusable Object-Oriented Software. Reading, Mass.: Addison-Wesley.
|
|
||||||
|
|
||||||
Garlan, D. and Shaw, M. (1993). 'An introduction to software architecture'. Advances in Software Engineering and Knowledge Engineering, 1 1-39.
|
|
||||||
|
|
||||||
Harold, E. R. and Means, W. S. (2002). XML in a Nutshell. Sebastopol. Calif.: O'Reilly.
|
|
||||||
|
|
||||||
Hofmeister, C., Nord, R. and Soni, D. (2000). Applied Software Architecture. Boston: AddisonWesley.
|
|
||||||
|
|
||||||
Hunter, D., Rafter, J., Fawcett, J. and Van Der Vlist, E. (2007). Beginning XML, 4th ed. Indianapolis, Ind.: Wrox Press.
|
|
||||||
|
|
||||||
Kircher, M. and Jain, P. (2004). Pattern-Oriented Software Architecture Volume 3: Patterns for Resource Management. New York: John Wiley & Sons.
|
|
||||||
|
|
||||||
Krutchen, P. (1995). 'The 4+1 view model of software architecture'. IEEE Software, 12 (6), 42-50.
|
|
||||||
|
|
||||||
Lange, C. F. J., Chaudron, M. R. V. and Muskens, J. (2006). 'UML software description and architecture description'. IEEE Software, 23 (2), 40-6.
|
|
||||||
|
|
||||||
Lewis, P. M., Bernstein, A. J. and Kifer, M. (2003). Databases and Transaction Processing: An Application-oriented Approach. Boston: Addison-Wesley.
|
|
||||||
|
|
||||||
Martin, D. and Sommerville, I. (2004). 'Patterns of interaction: Linking ethnomethodology and design'. ACM Trans. on Computer-Human Interaction, 11 (1), 59-89.
|
|
||||||
|
|
||||||
Nii, H. P . (1986). 'Blackboard systems, parts 1 and 2'. AI Magazine, 7 (3 and 4), 38-53 and 62-9.
|
|
||||||
|
|
||||||
Schmidt, D., Stal, M., Rohnert, H. and Buschmann, F. (2000). Pattern-Oriented Software Architecture Volume 2: Patterns for Concurrent and Networked Objects. New York: John Wiley & Sons.
|
|
||||||
|
|
||||||
Shaw, M. and Garlan, D. (1996). Software Architecture: Perspectives on an Emerging Discipline. Englewood Cliffs, NJ: Prentice Hall.
|
|
||||||
|
|
||||||
Usability group. (1998). 'Usability patterns'. Web publication. http://www.it.bton.ac.uk/cil/usability/patterns/.
|
|
||||||
Binary file not shown.
@@ -1,36 +0,0 @@
|
|||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from docling.datamodel.base_models import InputFormat
|
|
||||||
from docling.datamodel.pipeline_options import PdfPipelineOptions, AcceleratorOptions
|
|
||||||
from docling.document_converter import DocumentConverter, PdfFormatOption
|
|
||||||
|
|
||||||
def main():
|
|
||||||
input_doc_path = Path("se_arch.pdf")
|
|
||||||
|
|
||||||
pipeline_options = PdfPipelineOptions()
|
|
||||||
pipeline_options.do_ocr = True
|
|
||||||
pipeline_options.do_table_structure = True
|
|
||||||
accelerator_options = AcceleratorOptions(device="cpu")
|
|
||||||
|
|
||||||
converter = DocumentConverter(
|
|
||||||
format_options={
|
|
||||||
InputFormat.PDF: PdfFormatOption(
|
|
||||||
pipeline_options=pipeline_options,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
conversion_result = converter.convert(input_doc_path)
|
|
||||||
doc = conversion_result.document
|
|
||||||
|
|
||||||
md = doc.export_to_markdown()
|
|
||||||
|
|
||||||
output_path = Path("se_arch.md")
|
|
||||||
output_path.write_text(md, encoding="utf-8")
|
|
||||||
|
|
||||||
# doc_conversion_secs = conversion_result.timings["pipeline_total"].times
|
|
||||||
print(f"Saved markdown to {output_path}")
|
|
||||||
# print(f"Conversion secs: {doc_conversion_secs}")
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
73
secrets_template/.env.example
Normal file
73
secrets_template/.env.example
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
# Environment Variables Template
|
||||||
|
# Copy this file to `.env` and fill in real values.
|
||||||
|
# Do NOT commit `.env` — it is ignored by .gitignore.
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# Cloud LLM Gateway (Backend)
|
||||||
|
# ============================================================
|
||||||
|
|
||||||
|
# Vertex AI / Gemini
|
||||||
|
VERTEX_AI_PROJECT=vkist-project
|
||||||
|
VERTEX_AI_LOCATION=asia-southeast1
|
||||||
|
VERTEX_AI_GEMINI_ENDPOINT=https://asia-southeast1-aiplatform.googleapis.com/v1/projects/vkist-project/locations/asia-southeast1/publishers/google/models/gemini-2.5-pro:generateContent
|
||||||
|
VERTEX_AI_MODEL=medgemma
|
||||||
|
# GCP access token loaded from secrets/gcp_access_token.txt or via GCP_ACCESS_TOKEN_FILE
|
||||||
|
|
||||||
|
# MedGemma Modal endpoint (Ollama web server)
|
||||||
|
# Full /api/chat URL also accepted — base is normalized automatically.
|
||||||
|
MODAL_MEDGEMMA_ENDPOINT=https://dtj-tran--ollama-medgemma-ollamaserver-web.modal.run
|
||||||
|
MEDGEMMA_MODEL=medgemma:4b
|
||||||
|
PORT=8080
|
||||||
|
# MEDGEMMA_API_KEY loaded from secrets/modal_api_key.txt or via MEDGEMMA_API_KEY_FILE
|
||||||
|
# MODAL_API_KEY is required by the Modal SDK itself
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# CORS
|
||||||
|
# ============================================================
|
||||||
|
# Comma-separated list of allowed frontend origins
|
||||||
|
CORS_ORIGINS=http://localhost:3000,http://localhost:5173,http://localhost:4173
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# Storage / Triton
|
||||||
|
# ============================================================
|
||||||
|
TRITON_ENDPOINT=http://localhost:8080
|
||||||
|
TEMP_DIR=/tmp/analysis_jobs
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# Redis
|
||||||
|
# ============================================================
|
||||||
|
REDIS_HOST=localhost
|
||||||
|
REDIS_PORT=6379
|
||||||
|
REDIS_DB=0
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# Image Processing
|
||||||
|
# ============================================================
|
||||||
|
CLAHE_CLIP_LIMIT=2.0
|
||||||
|
CLAHE_TILE_SIZE=8,8
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# Exa Web Search (Agent Tools BFF)
|
||||||
|
# ============================================================
|
||||||
|
EXA_API_KEY=
|
||||||
|
# Canonical API guide: https://docs.exa.ai/reference/search-api-guide-for-coding-agents
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# Supabase — Knowledge Semantic Vector DB
|
||||||
|
# ============================================================
|
||||||
|
SUPABASE_URL=https://<project-ref>.supabase.co
|
||||||
|
SUPABASE_SERVICE_ROLE_KEY=
|
||||||
|
# Optional: authenticated read + RPC from backend with user JWT
|
||||||
|
SUPABASE_ANON_KEY=
|
||||||
|
# PoC only: return zero vector for supabase_query embedding (not for production RAG quality)
|
||||||
|
# EMBED_QUERY_MOCK=1
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# AWS S3 (if used)
|
||||||
|
# ============================================================
|
||||||
|
AWS_ACCESS_KEY_ID=
|
||||||
|
AWS_SECRET_ACCESS_KEY=
|
||||||
|
AWS_SESSION_TOKEN=
|
||||||
|
AWS_REGION=
|
||||||
|
AWS_DEFAULT_REGION=
|
||||||
|
AWS_ENDPOINT_URL=
|
||||||
170
secrets_template/README.md
Normal file
170
secrets_template/README.md
Normal file
@@ -0,0 +1,170 @@
|
|||||||
|
# Secrets Template — Developer Self-Setup Guide
|
||||||
|
|
||||||
|
This folder is the **tracked scaffold** for the local `secrets/` directory described in the [repository root README](../README.md#getting-started). It is safe to commit; it contains **no real credentials**.
|
||||||
|
|
||||||
|
Each developer creates and maintains their own `secrets/` folder at the repository root. That folder is **gitignored** and never pushed to Git.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
From the repository root (`PILOT_PROJECT/`):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Create your local secrets directory
|
||||||
|
mkdir -p secrets/aws_secret
|
||||||
|
|
||||||
|
# 2. Copy the environment template
|
||||||
|
cp secrets_template/.env.example secrets/aws_secret/.env
|
||||||
|
|
||||||
|
# 3. Create plaintext secret files (see tables below)
|
||||||
|
touch secrets/gcp_access_token.txt secrets/modal_api_key.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
Then:
|
||||||
|
|
||||||
|
1. Open `secrets/aws_secret/.env` and fill in values for the services you need.
|
||||||
|
2. Paste single-value secrets into the corresponding `.txt` files (one secret per file, no quotes, no trailing newline required).
|
||||||
|
3. Confirm `secrets/` is ignored: `git check-ignore -v secrets/` should report a match.
|
||||||
|
|
||||||
|
Optional — add a local pointer file (also gitignored):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp secrets_template/README.md secrets/README.md
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Expected Layout
|
||||||
|
|
||||||
|
After setup, your local tree should look like this:
|
||||||
|
|
||||||
|
```
|
||||||
|
PILOT_PROJECT/
|
||||||
|
├── secrets/ # gitignored — your machine only
|
||||||
|
│ ├── gcp_access_token.txt # Vertex AI / Gemini bearer token
|
||||||
|
│ ├── modal_api_key.txt # MedGemma Modal endpoint API key
|
||||||
|
│ ├── aws_secret/
|
||||||
|
│ │ └── .env # consolidated env vars (Supabase, Exa, AWS, Modal SDK, …)
|
||||||
|
│ ├── key # optional — SSH private key (infra)
|
||||||
|
│ ├── key.pub # optional — SSH public key
|
||||||
|
│ ├── init_ssh.sh # optional — SSH setup helper
|
||||||
|
│ └── server_vkist.sh # optional — server bootstrap script
|
||||||
|
└── secrets_template/ # tracked — templates & docs only
|
||||||
|
├── README.md # this file
|
||||||
|
├── .env.example # copy → secrets/aws_secret/.env
|
||||||
|
└── gemini_api_key.txt # production reference (not a local key file)
|
||||||
|
```
|
||||||
|
|
||||||
|
Only create the files you need for the components you are working on. Missing secrets for unused services will not block unrelated work.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Secret Inventory
|
||||||
|
|
||||||
|
### Plaintext files (`secrets/` root)
|
||||||
|
|
||||||
|
| File | Env var | Used by | How to obtain |
|
||||||
|
|------|---------|---------|---------------|
|
||||||
|
| `gcp_access_token.txt` | `GCP_ACCESS_TOKEN` | Backend Cloud LLM Gateway (Vertex AI / Gemini) | `gcloud auth print-access-token` (short-lived; refresh as needed) |
|
||||||
|
| `modal_api_key.txt` | `MEDGEMMA_API_KEY` | Backend + Modal MedGemma endpoint auth | Project lead or Modal dashboard → API keys |
|
||||||
|
|
||||||
|
**Override:** point to a file outside `secrets/` with `{NAME}_FILE`, e.g. `GCP_ACCESS_TOKEN_FILE=/path/to/token.txt`. The loader checks `*_FILE` first, then `secrets/<filename>`.
|
||||||
|
|
||||||
|
Reference: `workspace/sprint_1_2/CODEBASE/backend/implementation/config.py`
|
||||||
|
|
||||||
|
### Environment bundle (`secrets/aws_secret/.env`)
|
||||||
|
|
||||||
|
Copy from `secrets_template/.env.example`. This file is auto-loaded by agent-tool smoke tests and the knowledge ingestion pipeline when present.
|
||||||
|
|
||||||
|
| Variable | Required for | Notes |
|
||||||
|
|----------|--------------|-------|
|
||||||
|
| `SUPABASE_URL` | Knowledge RAG, agent `supabase_query` | Supabase project URL |
|
||||||
|
| `SUPABASE_SERVICE_ROLE_KEY` | Backend writes / ingestion upload | Dashboard → Project Settings → API |
|
||||||
|
| `SUPABASE_PUBLISHABLE_KEY` / `SUPABASE_ANON_KEY` | Client-side or read-only flows | Same dashboard section |
|
||||||
|
| `SUPABASE_DB_URL` | Ingestion pipeline (PostgreSQL path) | Dashboard → Database → Connection string |
|
||||||
|
| `EXA_API_KEY` | Agent `exa_search` tool | [Exa dashboard](https://docs.exa.ai/) |
|
||||||
|
| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION` | S3 uploads, corpus storage | IAM user or role credentials |
|
||||||
|
| `MODAL_KEY`, `MODAL_SECRET` | Modal SDK CLI / deploys | Modal account settings |
|
||||||
|
| `HF_TOKEN` | Hugging Face model downloads | huggingface.co → Settings → Access Tokens |
|
||||||
|
| `FIGMA_ACCESS_TOKEN` | Design automation (optional) | Figma → Personal access tokens |
|
||||||
|
|
||||||
|
Non-secret configuration (endpoints, ports, CORS) lives in `workspace/sprint_1_2/CODEBASE/.env.example` — copy that to `CODEBASE/.env` separately; do **not** commit either `.env` file.
|
||||||
|
|
||||||
|
Reference paths:
|
||||||
|
|
||||||
|
- `workspace/sprint_1_2/CODEBASE/knowledge/implementation/ingestion/config.py` → `secrets/aws_secret/.env`
|
||||||
|
- `workspace/sprint_1_2/CODEBASE/backend/tests/smoke_agent_tools_server.py` → same path
|
||||||
|
- `workspace/sprint_1_2/CODEBASE/ml/tests/agent_tools/README.md` → smoke test auto-load
|
||||||
|
|
||||||
|
### Optional infra files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
|------|---------|
|
||||||
|
| `key` / `key.pub` | SSH key pair for remote servers or CI |
|
||||||
|
| `init_ssh.sh` | Helper to register SSH keys with `ssh-agent` |
|
||||||
|
| `server_vkist.sh` | Server-side bootstrap (project-specific) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Gemini API Key (Production Reference)
|
||||||
|
|
||||||
|
`secrets_template/gemini_api_key.txt` documents where the **production** Gemini key lives (GCP Secret Manager: `gemini-api-key` in project `vkist-project`). It is **not** a file you paste a key into for local dev.
|
||||||
|
|
||||||
|
For local backend work, use `gcp_access_token.txt` (Vertex AI bearer token) instead.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Verify Your Setup
|
||||||
|
|
||||||
|
### Backend secrets (GCP + Modal)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd workspace/sprint_1_2/CODEBASE
|
||||||
|
conda activate vkist_ultra # or your project env
|
||||||
|
PYTHONPATH=. python -c "from backend.implementation import config; print('GCP token loaded:', bool(config.GCP_ACCESS_TOKEN))"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Agent tools smoke (Exa + Supabase, no GCP)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd workspace/sprint_1_2/CODEBASE
|
||||||
|
PYTHONPATH=. python backend/tests/smoke_agent_tools_server.py
|
||||||
|
# In another terminal:
|
||||||
|
cd workspace/sprint_1_2/CODEBASE/ml/tests/agent_tools && npm run smoke:bff
|
||||||
|
```
|
||||||
|
|
||||||
|
### Knowledge ingestion
|
||||||
|
|
||||||
|
See `workspace/sprint_1_2/CODEBASE/knowledge/implementation/ingestion/README.md` — requires `secrets/aws_secret/.env` with Supabase credentials.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Security Rules
|
||||||
|
|
||||||
|
1. **Never commit** `secrets/` or any file containing real keys, tokens, or passwords.
|
||||||
|
2. The root `.gitignore` already excludes `secrets/`. Before adding new secret paths, confirm they remain ignored.
|
||||||
|
3. Do not paste secrets into source code, logs, issues, or PR comments.
|
||||||
|
4. Share credentials only through your team's approved secret manager (1Password, Vault, GCP Secret Manager, etc.).
|
||||||
|
5. Rotate tokens if they are ever exposed. GCP access tokens are short-lived; treat `.env` and API keys as long-lived secrets.
|
||||||
|
|
||||||
|
For agent and CI guardrails, see `AGENT_SKILL/secrets_and_phi_safety/SKILL.md`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Getting Credentials
|
||||||
|
|
||||||
|
If you do not have access to a required service:
|
||||||
|
|
||||||
|
1. Ask the project lead for onboarding to the relevant cloud account (GCP, Modal, Supabase, AWS).
|
||||||
|
2. Use the tables above to identify which variables or files you need for your sprint task.
|
||||||
|
3. Start with the smoke tests in [Verify Your Setup](#verify-your-setup) to confirm only the secrets you need are wired correctly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Related Documentation
|
||||||
|
|
||||||
|
- [Repository README — Secrets Management](../README.md#secrets-management-important)
|
||||||
|
- [CODEBASE `.env.example`](../workspace/sprint_1_2/CODEBASE/.env.example) — non-secret runtime configuration
|
||||||
|
- [Agent tools smoke tests](../workspace/sprint_1_2/CODEBASE/ml/tests/agent_tools/README.md)
|
||||||
|
- [Knowledge ingestion prerequisites](../workspace/sprint_1_2/CODEBASE/knowledge/implementation/ingestion/README.md)
|
||||||
14
secrets_template/gemini_api_key.txt
Normal file
14
secrets_template/gemini_api_key.txt
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
# Gemini API Key Reference
|
||||||
|
# This file documents the secret reference for the Gemini API key used by the Cloud LLM Gateway.
|
||||||
|
#
|
||||||
|
# Production secret location: GCP Secret Manager
|
||||||
|
# Secret name: gemini-api-key
|
||||||
|
# Project: vkist-project
|
||||||
|
# Region: asia-southeast1
|
||||||
|
#
|
||||||
|
# Access pattern:
|
||||||
|
# FastAPI Cloud LLM Gateway fetches key from GCP Secret Manager at startup
|
||||||
|
# Key never written to disk; accessed via GCP Workload Identity or service account
|
||||||
|
#
|
||||||
|
# Fallback: modal.Secret.from_name("gcp-secrets") in Modal deployment
|
||||||
|
# allows MedGemma container to authenticate to GCP Vertex AI if needed.
|
||||||
@@ -11,15 +11,11 @@ Knee Ultrasound Analysis API được xây dựng bằng FastAPI, chuyên phục
|
|||||||
|
|
||||||
### 1.1 Các chức năng chính
|
### 1.1 Các chức năng chính
|
||||||
|
|
||||||
* **Phân loại góc chụp siêu âm (Angle Classification):** Tự động nhận dạng mặt cắt/góc chụp từ ảnh đầu vào bao gồm các nhãn: `med-lat`, `post-trans`, `sup-trans-flex`, và `sup-up-long`.
|
- **Phân loại góc chụp siêu âm (Angle Classification):** Tự động nhận dạng mặt cắt/góc chụp từ ảnh đầu vào bao gồm các nhãn: `med-lat`, `post-trans`, `sup-trans-flex`, và `sup-up-long`.
|
||||||
|
- **Phát hiện viêm (Inflammation Detection):** Xác định sự hiện diện của tình trạng viêm khớp gối qua hai góc chụp chính là `sup-up-long` và `post-trans`.
|
||||||
* **Phát hiện viêm (Inflammation Detection):** Xác định sự hiện diện của tình trạng viêm khớp gối qua hai góc chụp chính là `sup-up-long` và `post-trans`.
|
- **Phân đoạn ảnh ngữ nghĩa (Segmentation):** Tách biệt các cấu trúc giải phẫu đích (dịch khớp, gân, xương, màng hoạt dịch...) thành các phân vùng mặt nạ màu riêng biệt.
|
||||||
|
- **Đo độ dày tự động (Thickness Measurement):** Tự động tính toán khoảng cách hình học theo đơn vị milimét ($mm$) giữa các phân vùng mô mềm đã được phân đoạn (chỉ áp dụng đối với mặt cắt góc `sup-up-long`).
|
||||||
* **Phân đoạn ảnh ngữ nghĩa (Segmentation):** Tách biệt các cấu trúc giải phẫu đích (dịch khớp, gân, xương, màng hoạt dịch...) thành các phân vùng mặt nạ màu riêng biệt.
|
- **Đánh giá mức độ viêm (Severity Analysis):** Xếp hạng thang điểm mức độ nghiêm trọng của viêm từ cấp độ 0 (Rất nhẹ) đến cấp độ 3 (Nặng) dựa trên tỷ lệ diện tích dịch khớp và sự tăng sinh màng hoạt dịch.
|
||||||
|
|
||||||
* **Đo độ dày tự động (Thickness Measurement):** Tự động tính toán khoảng cách hình học theo đơn vị milimét ($mm$) giữa các phân vùng mô mềm đã được phân đoạn (chỉ áp dụng đối với mặt cắt góc `sup-up-long`).
|
|
||||||
|
|
||||||
* **Đánh giá mức độ viêm (Severity Analysis):** Xếp hạng thang điểm mức độ nghiêm trọng của viêm từ cấp độ 0 (Rất nhẹ) đến cấp độ 3 (Nặng) dựa trên tỷ lệ diện tích dịch khớp và sự tăng sinh màng hoạt dịch.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -27,11 +23,13 @@ Knee Ultrasound Analysis API được xây dựng bằng FastAPI, chuyên phục
|
|||||||
|
|
||||||
Khi nhận tập tin ảnh từ máy trạm (Client), hệ thống sẽ thực hiện phân nhánh xử lý logic động dựa trên kết quả của khối phân loại góc chụp:
|
Khi nhận tập tin ảnh từ máy trạm (Client), hệ thống sẽ thực hiện phân nhánh xử lý logic động dựa trên kết quả của khối phân loại góc chụp:
|
||||||
|
|
||||||
|
|
||||||
| Góc chụp phát hiện | Quy trình xử lý chi tiết trong Backend pipeline |
|
| Góc chụp phát hiện | Quy trình xử lý chi tiết trong Backend pipeline |
|
||||||
| --- | --- |
|
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| **`post-trans`** | Phân loại góc $\rightarrow$ Phát hiện viêm $\rightarrow$ Phân đoạn ảnh POST $\rightarrow$ Trả kết quả JSON & Mask.|
|
| `post-trans` | Phân loại góc $\rightarrow$ Phát hiện viêm $\rightarrow$ Phân đoạn ảnh POST $\rightarrow$ Trả kết quả JSON & Mask. |
|
||||||
| **`sup-up-long`** | Phân loại góc $\rightarrow$ Phát hiện viêm $\rightarrow$ Phân đoạn ảnh SUP $\rightarrow$ Đo độ dày mô $\rightarrow$ Đánh giá mức độ nặng $\rightarrow$ Trả kết quả.|
|
| `sup-up-long` | Phân loại góc $\rightarrow$ Phát hiện viêm $\rightarrow$ Phân đoạn ảnh SUP $\rightarrow$ Đo độ dày mô $\rightarrow$ Đánh giá mức độ nặng $\rightarrow$ Trả kết quả. |
|
||||||
| **`med-lat`** or **`sup-trans-flex`** | Chỉ thực hiện phân loại góc $\rightarrow$ Trả kết quả trực tiếp (Bỏ qua nhánh phân đoạn & đo lường).|
|
| `med-lat` or `sup-trans-flex` | Chỉ thực hiện phân loại góc $\rightarrow$ Trả kết quả trực tiếp (Bỏ qua nhánh phân đoạn & đo lường). |
|
||||||
|
|
||||||
|
|
||||||
```plantuml
|
```plantuml
|
||||||
@startuml
|
@startuml
|
||||||
@@ -70,19 +68,27 @@ stop
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 2. Hướng dẫn cài đặt & Triển khai môi trường
|
## 2. Hướng dẫn cài đặt & Triển khai môi trường
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 2.1 Yêu cầu hệ thống tối thiểu
|
### 2.1 Yêu cầu hệ thống tối thiểu
|
||||||
|
|
||||||
|
|
||||||
| Thành phần cấu phần | Thông số kỹ thuật yêu cầu tối thiểu |
|
| Thành phần cấu phần | Thông số kỹ thuật yêu cầu tối thiểu |
|
||||||
| --- | --- |
|
| ------------------------- | ----------------------------------------------------------------------------- |
|
||||||
| **Hệ điều hành** | Ubuntu 20.04+ / Windows 10+ / macOS 12+ |
|
| **Hệ điều hành** | Ubuntu 20.04+ / Windows 10+ / macOS 12+ |
|
||||||
| **Môi trường Python** | Phiên bản 3.10 cố định |
|
| **Môi trường Python** | Phiên bản 3.10 cố định |
|
||||||
| **Bộ nhớ RAM** | 16 GB trở lên |
|
| **Bộ nhớ RAM** | 16 GB trở lên |
|
||||||
| **Bộ xử lý đồ họa (GPU)** | NVIDIA GPU hỗ trợ nền tảng CUDA 12.4 (Khuyến nghị để tối ưu tốc độ) |
|
| **Bộ xử lý đồ họa (GPU)** | NVIDIA GPU hỗ trợ nền tảng CUDA 12.4 (Khuyến nghị để tối ưu tốc độ) |
|
||||||
| **Dung lượng VRAM** | Tối thiểu 8 GB (Khuyến nghị 16 GB nếu chạy song song đồng thời nhiều mô hình)|
|
| **Dung lượng VRAM** | Tối thiểu 8 GB (Khuyến nghị 16 GB nếu chạy song song đồng thời nhiều mô hình) |
|
||||||
| **Ổ cứng lưu trữ** | Tối thiểu 15 GB dung lượng trống (Dành cho bộ cài đặt và file weights `.pth`) |
|
| **Ổ cứng lưu trữ** | Tối thiểu 15 GB dung lượng trống (Dành cho bộ cài đặt và file weights `.pth`) |
|
||||||
| **Bộ công cụ bổ trợ** | CUDA Toolkit 12.4 & cuDNN 9.x tương thích tương ứng|
|
| **Bộ công cụ bổ trợ** | CUDA Toolkit 12.4 & cuDNN 9.x tương thích tương ứng |
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 2.2 Khởi tạo môi trường ảo
|
### 2.2 Khởi tạo môi trường ảo
|
||||||
|
|
||||||
@@ -102,7 +108,7 @@ conda activate vkist-ultrasound
|
|||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
*Hoặc khởi tạo nhanh bằng mô-đun thư viện chuẩn `venv` nếu hệ thống chưa cài đặt Anaconda*:
|
*Hoặc khởi tạo nhanh bằng mô-đun thư viện chuẩn* `venv` *nếu hệ thống chưa cài đặt Anaconda*:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Trên nền tảng hệ điều hành Linux / macOS
|
# Trên nền tảng hệ điều hành Linux / macOS
|
||||||
@@ -114,6 +120,8 @@ venv\Scripts\activate
|
|||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 2.3 Cài đặt các gói thư viện phụ thuộc (Dependencies)
|
### 2.3 Cài đặt các gói thư viện phụ thuộc (Dependencies)
|
||||||
|
|
||||||
Thực hiện cài đặt các thư viện lõi quy định trong tệp cấu hình:
|
Thực hiện cài đặt các thư viện lõi quy định trong tệp cấu hình:
|
||||||
@@ -133,8 +141,8 @@ pip install torch==2.5.0+cu124 torchvision==0.20.0+cu124 --index-url https://dow
|
|||||||
> ⚠️ **LƯU Ý QUAN TRỌNG VỀ PACKAGE NATTEN:**
|
> ⚠️ **LƯU Ý QUAN TRỌNG VỀ PACKAGE NATTEN:**
|
||||||
> Dòng cấu hình cài đặt gói `natten==0.17.3+torch250cu124` mặc định đã bị gắn chú thích (`#` comment out) trong tệp `requirements.txt`. Nếu bạn sử dụng các kiến trúc mạng Transformer nâng cao yêu cầu gói này, bắt buộc cài đặt thủ công qua liên kết phân phối bánh xe (wheels) chính thức:
|
> Dòng cấu hình cài đặt gói `natten==0.17.3+torch250cu124` mặc định đã bị gắn chú thích (`#` comment out) trong tệp `requirements.txt`. Nếu bạn sử dụng các kiến trúc mạng Transformer nâng cao yêu cầu gói này, bắt buộc cài đặt thủ công qua liên kết phân phối bánh xe (wheels) chính thức:
|
||||||
> `pip install natten==0.17.3+torch250cu124 -f https://shi-labs.com/natten/wheels/`
|
> `pip install natten==0.17.3+torch250cu124 -f https://shi-labs.com/natten/wheels/`
|
||||||
>
|
|
||||||
>
|
|
||||||
|
|
||||||
### 2.4 Cấu trúc cây thư mục dự án chuẩn
|
### 2.4 Cấu trúc cây thư mục dự án chuẩn
|
||||||
|
|
||||||
@@ -168,6 +176,8 @@ project/
|
|||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 2.5 Khởi động máy chủ dịch vụ
|
### 2.5 Khởi động máy chủ dịch vụ
|
||||||
|
|
||||||
Thực thi lệnh chạy máy chủ tại thư mục gốc:
|
Thực thi lệnh chạy máy chủ tại thư mục gốc:
|
||||||
@@ -187,26 +197,23 @@ python app.py
|
|||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
* **Giao diện Web UI kiểm thử trực quan:** `http://localhost:8000`
|
- **Giao diện Web UI kiểm thử trực quan:** `http://localhost:8000`
|
||||||
|
- **Tài liệu API tương tác tự động (Swagger UI):** `http://localhost:8000/docs`
|
||||||
|
|
||||||
* **Tài liệu API tương tác tự động (Swagger UI):** `http://localhost:8000/docs`
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 3. Tài liệu đặc tả API (API Reference)
|
## 3. Tài liệu đặc tả API (API Reference)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 3.1 Trạng thái hoạt động (Health Check)
|
### 3.1 Trạng thái hoạt động (Health Check)
|
||||||
|
|
||||||
* **Endpoint:** `GET /api/health`
|
- **Endpoint:** `GET /api/health`
|
||||||
|
- **Chức năng:** Kiểm tra tính sẵn sàng phục vụ của cụm dịch vụ API Backend.
|
||||||
|
- **Định dạng dữ liệu phản hồi (Response JSON):**
|
||||||
|
|
||||||
|
|
||||||
* **Chức năng:** Kiểm tra tính sẵn sàng phục vụ của cụm dịch vụ API Backend.
|
|
||||||
|
|
||||||
|
|
||||||
* **Định dạng dữ liệu phản hồi (Response JSON):**
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"status": "healthy"
|
"status": "healthy"
|
||||||
@@ -222,28 +229,36 @@ Mã hóa dữ liệu đầu vào dưới dạng tệp tin `multipart/form-data`
|
|||||||
|
|
||||||
#### Các tham số yêu cầu (Request Parameters)
|
#### Các tham số yêu cầu (Request Parameters)
|
||||||
|
|
||||||
|
|
||||||
| Tham số cấu hình | Phương thức truyền | Kiểu dữ liệu | Giá trị mặc định | Định nghĩa chức năng chi tiết |
|
| Tham số cấu hình | Phương thức truyền | Kiểu dữ liệu | Giá trị mặc định | Định nghĩa chức năng chi tiết |
|
||||||
| --- | --- | --- | --- | --- |
|
| -------------------- | ------------------ | ------------ | --------------------- | ----------------------------------------------------------------------------------------- |
|
||||||
| **`image`** | Multipart Form | Binary File | *Bắt buộc* | Tệp tin ảnh siêu âm đầu gối cần xử lý (Hỗ trợ mở rộng định dạng: `.jpg`, `.png`, `.bmp`).|
|
| `image` | Multipart Form | Binary File | *Bắt buộc* | Tệp tin ảnh siêu âm đầu gối cần xử lý (Hỗ trợ mở rộng định dạng: `.jpg`, `.png`, `.bmp`). |
|
||||||
| **`angle_model`** | Query String | String | `convnext` | Tên định danh mô hình đảm nhận tác vụ phân loại góc chụp.|
|
| `angle_model` | Query String | String | `convnext` | Tên định danh mô hình đảm nhận tác vụ phân loại góc chụp. |
|
||||||
| **`inflammation_model`** | Query String | String | `efficientnet_b0` | Mô hình phát hiện tình trạng viêm (Hiện tại cố định cấu hình mạng).|
|
| `inflammation_model` | Query String | String | `efficientnet_b0` | Mô hình phát hiện tình trạng viêm (Hiện tại cố định cấu hình mạng). |
|
||||||
| **`segment_model_sup`** | Query String | String | `deeplabv3` | Mô hình phân đoạn cấu trúc giải phẫu dành cho mặt cắt góc `sup-up-long`.|
|
| `segment_model_sup` | Query String | String | `deeplabv3` | Mô hình phân đoạn cấu trúc giải phẫu dành cho mặt cắt góc `sup-up-long`. |
|
||||||
| **`segment_model_post`** | Query String | String | `deeplabv3_resnet101` | Mô hình phân đoạn cấu trúc giải phẫu dành cho mặt cắt góc `post-trans`.|
|
| `segment_model_post` | Query String | String | `deeplabv3_resnet101` | Mô hình phân đoạn cấu trúc giải phẫu dành cho mặt cắt góc `post-trans`. |
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
#### Danh sách định danh mô hình khả dụng trong hệ thống
|
#### Danh sách định danh mô hình khả dụng trong hệ thống
|
||||||
|
|
||||||
|
|
||||||
| Phân nhóm Task | Tên tham số truyền vào | Kiến trúc mạng nơ-ron gốc | Mô tả đặc tính đầu ra |
|
| Phân nhóm Task | Tên tham số truyền vào | Kiến trúc mạng nơ-ron gốc | Mô tả đặc tính đầu ra |
|
||||||
| --- | --- | --- | --- |
|
| ---------------------- | ---------------------- | ------------------------- | ------------------------------------------------------ |
|
||||||
| **Phân loại Góc chụp** | `convnext` | ConvNeXt Tiny | Phân cấp phân loại ra 4 lớp nhãn đầu ra.|
|
| **Phân loại Góc chụp** | `convnext` | ConvNeXt Tiny | Phân cấp phân loại ra 4 lớp nhãn đầu ra. |
|
||||||
| | `densenet` | DenseNet-121 | Mạng kết nối dày đặc.|
|
| | `densenet` | DenseNet-121 | Mạng kết nối dày đặc. |
|
||||||
| | `resnet50` | ResNet-50 | Kiến trúc mạng dư thừa tiêu chuẩn.|
|
| | `resnet50` | ResNet-50 | Kiến trúc mạng dư thừa tiêu chuẩn. |
|
||||||
| | `efficientnet_b2` | EfficientNet-B2 | Tối ưu hóa đa quy mô tài nguyên mạng.|
|
| | `efficientnet_b2` | EfficientNet-B2 | Tối ưu hóa đa quy mô tài nguyên mạng. |
|
||||||
| | `swin` | Swin Transformer V2-S | Kiến trúc Attention cửa sổ dịch chuyển.|
|
| | `swin` | Swin Transformer V2-S | Kiến trúc Attention cửa sổ dịch chuyển. |
|
||||||
| **Phân đoạn góc SUP** | `deeplabv3` | DeepLabV3 ResNet-50 | Trích xuất đặc trưng đa tỷ lệ với 7 lớp đầu ra.|
|
| **Phân đoạn góc SUP** | `deeplabv3` | DeepLabV3 ResNet-50 | Trích xuất đặc trưng đa tỷ lệ với 7 lớp đầu ra. |
|
||||||
| | `unet_resnet101` | UNet + ResNet-101 | Kiến trúc Encoder-Decoder kết hợp ResNet.|
|
| | `unet_resnet101` | UNet + ResNet-101 | Kiến trúc Encoder-Decoder kết hợp ResNet. |
|
||||||
| | `efficientfeedback` | EfficientFeedbackNetwork | Thiết kế tùy biến riêng có liên kết phản hồi dữ liệu.|
|
| | `efficientfeedback` | EfficientFeedbackNetwork | Thiết kế tùy biến riêng có liên kết phản hồi dữ liệu. |
|
||||||
| | `unet3plus` | UNet3+ with Attention | Cơ chế Attention kết hợp kết nối toàn diện Full-scale.|
|
| | `unet3plus` | UNet3+ with Attention | Cơ chế Attention kết hợp kết nối toàn diện Full-scale. |
|
||||||
| **Phân đoạn góc POST** | `deeplabv3_resnet101` | DeepLabV3 ResNet-101 | Cấu trúc chuyên sâu phân đoạn góc nhìn mặt sau.|
|
| **Phân đoạn góc POST** | `deeplabv3_resnet101` | DeepLabV3 ResNet-101 | Cấu trúc chuyên sâu phân đoạn góc nhìn mặt sau. |
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
#### Cấu trúc dữ liệu JSON phản hồi (Response Body Schema)
|
#### Cấu trúc dữ liệu JSON phản hồi (Response Body Schema)
|
||||||
|
|
||||||
@@ -300,9 +315,11 @@ Mã hóa dữ liệu đầu vào dưới dạng tệp tin `multipart/form-data`
|
|||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
#### Ví dụ mã triển khai gọi dịch vụ (Client Invocations)
|
#### Ví dụ mã triển khai gọi dịch vụ (Client Invocations)
|
||||||
|
|
||||||
* **Sử dụng lệnh Client cURL CLI:**
|
- **Sử dụng lệnh Client cURL CLI:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
curl -X POST "http://localhost:8000/api/analyze?angle_model=convnext&segment_model_sup=deeplabv3" \
|
curl -X POST "http://localhost:8000/api/analyze?angle_model=convnext&segment_model_sup=deeplabv3" \
|
||||||
@@ -310,7 +327,7 @@ curl -X POST "http://localhost:8000/api/analyze?angle_model=convnext&segment_mod
|
|||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
* **Triển khai ứng dụng gọi qua script Python (Requests):**
|
- **Triển khai ứng dụng gọi qua script Python (Requests):**
|
||||||
|
|
||||||
```python
|
```python
|
||||||
import requests
|
import requests
|
||||||
@@ -334,53 +351,71 @@ print("Số liệu đo lường hình học:", parsed_result.get("measurement"))
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 4. Các thông số cấu hình lõi hệ thống
|
## 4. Các thông số cấu hình lõi hệ thống
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 4.1 Hằng số hệ thống trong `app.py`
|
### 4.1 Hằng số hệ thống trong `app.py`
|
||||||
|
|
||||||
|
|
||||||
| Tên định danh hằng số | Giá trị mặc định | Diễn giải chức năng kỹ thuật |
|
| Tên định danh hằng số | Giá trị mặc định | Diễn giải chức năng kỹ thuật |
|
||||||
| --- | --- | --- |
|
| --------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `UPLOAD_FOLDER` | `'uploads'` | Đường dẫn cục bộ lưu trữ file ảnh thô nhận từ máy trạm.|
|
| `UPLOAD_FOLDER` | `'uploads'` | Đường dẫn cục bộ lưu trữ file ảnh thô nhận từ máy trạm. |
|
||||||
| `RESULTS_FOLDER` | `'results'` | Đường dẫn lưu ảnh màu sau phân đoạn (Color Mask Overlayed).|
|
| `RESULTS_FOLDER` | `'results'` | Đường dẫn lưu ảnh màu sau phân đoạn (Color Mask Overlayed). |
|
||||||
| `TEMPLATES_FOLDER` | `'templates'` | Thư mục chứa mã nguồn giao diện phân tích Web UI.|
|
| `TEMPLATES_FOLDER` | `'templates'` | Thư mục chứa mã nguồn giao diện phân tích Web UI. |
|
||||||
| `PIXEL_TO_MM` | $\frac{45.0}{655.0} \approx 0.0687$ | Hệ số chuyển đổi từ độ phân giải pixel sang kích thước thực tế ($mm$). Phụ thuộc cố định vào cấu hình đầu ra của phần cứng máy quét siêu âm.|
|
| `PIXEL_TO_MM` | $\frac{45.0}{655.0} \approx 0.0687$ | Hệ số chuyển đổi từ độ phân giải pixel sang kích thước thực tế ($mm$). Phụ thuộc cố định vào cấu hình đầu ra của phần cứng máy quét siêu âm. |
|
||||||
| `DEFAULT_MEASURE_IDS` | `[1, 5]` | Danh sách mảng chứa ID nhãn lớp cấu trúc giải phẫu kích hoạt thuật toán đo độ dày: `1 = effusion` (Dịch khớp), `5 = synovium` (Màng hoạt dịch).|
|
| `DEFAULT_MEASURE_IDS` | `[1, 5]` | Danh sách mảng chứa ID nhãn lớp cấu trúc giải phẫu kích hoạt thuật toán đo độ dày: `1 = effusion` (Dịch khớp), `5 = synovium` (Màng hoạt dịch). |
|
||||||
| `device` | `cuda` hoặc `cpu` | Khối phần cứng thực thi tính toán đồ họa (Tự động thiết lập dựa trên tính khả dụng của driver NVIDIA).|
|
| `device` | `cuda` hoặc `cpu` | Khối phần cứng thực thi tính toán đồ họa (Tự động thiết lập dựa trên tính khả dụng của driver NVIDIA). |
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 4.2 Cấu hình Pipeline tiền xử lý và biến đổi ma trận ảnh (Transforms)
|
### 4.2 Cấu hình Pipeline tiền xử lý và biến đổi ma trận ảnh (Transforms)
|
||||||
|
|
||||||
Hệ thống phân tách ảnh đầu vào thành các luồng biến đổi riêng biệt trước khi nạp vào tensor mô hình tùy thuộc vào mục tiêu xử lý chuyên biệt:
|
Hệ thống phân tách ảnh đầu vào thành các luồng biến đổi riêng biệt trước khi nạp vào tensor mô hình tùy thuộc vào mục tiêu xử lý chuyên biệt:
|
||||||
|
|
||||||
|
|
||||||
| Luồng xử lý Pipeline ảnh | Kích thước chuyển đổi (Resize) | Quy định chuẩn hóa phân phối ma trận (Normalization) |
|
| Luồng xử lý Pipeline ảnh | Kích thước chuyển đổi (Resize) | Quy định chuẩn hóa phân phối ma trận (Normalization) |
|
||||||
| --- | --- | --- |
|
| ------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------- |
|
||||||
| **Phân loại góc & Phát hiện viêm** | <br>$224 \times 224$ pixel | Áp dụng phân phối phân cấp:<br> $\text{mean} = [0.485, 0.456, 0.406]$, <br> $\text{std} = [0.229, 0.224, 0.225]$ |
|
| **Phân loại góc & Phát hiện viêm** | $224 \times 224$ pixel | Áp dụng phân phối phân cấp: $\text{mean} = [0.485, 0.456, 0.406]$, $\text{std} = [0.229, 0.224, 0.225]$ |
|
||||||
| **Phân đoạn cấu trúc (Segmentation)** | <br>$512 \times 512$ pixel | Không áp dụng chuẩn hóa phân phối (Chỉ thực thi hàm chuyển đổi tensor `ToTensor()`) |
|
| **Phân đoạn cấu trúc (Segmentation)** | $512 \times 512$ pixel | Không áp dụng chuẩn hóa phân phối (Chỉ thực thi hàm chuyển đổi tensor `ToTensor()`) |
|
||||||
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 5. Ràng buộc kỹ thuật & Quy tắc thiết kế hệ thống
|
## 5. Ràng buộc kỹ thuật & Quy tắc thiết kế hệ thống
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 5.1 Quản lý và giải phóng tài nguyên bộ nhớ GPU (VRAM Leak Warning)
|
### 5.1 Quản lý và giải phóng tài nguyên bộ nhớ GPU (VRAM Leak Warning)
|
||||||
|
|
||||||
Trong phiên bản hiện tại, logic xử lý nội tại của API kích hoạt các hàm `load_angle_model()`, `load_inflammation_model()`, và `load_segmentation_model_*()` trực tiếp bên trong vòng đời của mỗi phiên request nhận về. Hành vi này ép buộc GPU liên tục nạp lại dữ liệu tệp `.pth` vào VRAM cho mỗi giao dịch HTTP, sinh ra độ trễ (Overhead) I/O lớn và tiềm ẩn nguy cơ tràn bộ nhớ hệ thống. Khi triển khai môi trường Production, bắt buộc phải tái cấu trúc chuyển các hàm này thành Singleton dịch vụ (Tải một lần duy nhất lúc khởi động tiến trình Web Server).
|
Trong phiên bản hiện tại, logic xử lý nội tại của API kích hoạt các hàm `load_angle_model()`, `load_inflammation_model()`, và `load_segmentation_model_*()` trực tiếp bên trong vòng đời của mỗi phiên request nhận về. Hành vi này ép buộc GPU liên tục nạp lại dữ liệu tệp `.pth` vào VRAM cho mỗi giao dịch HTTP, sinh ra độ trễ (Overhead) I/O lớn và tiềm ẩn nguy cơ tràn bộ nhớ hệ thống. Khi triển khai môi trường Production, bắt buộc phải tái cấu trúc chuyển các hàm này thành Singleton dịch vụ (Tải một lần duy nhất lúc khởi động tiến trình Web Server).
|
||||||
|
|
||||||
### 5.2 Ràng buộc phi tuyến tính của tham số vật lý `PIXEL_TO_MM`
|
### 5.2 Ràng buộc phi tuyến tính của tham số vật lý `PIXEL_TO_MM`
|
||||||
|
|
||||||
Hằng số quy đổi $\text{PIXEL\_TO\_MM} = \frac{45.0}{655.0}$ là một giá trị được cấu hình cứng (Hardcoded) trong mã nguồn, đặc trưng duy nhất cho một dòng máy siêu âm lâm sàng có tỷ lệ hiển thị $45mm$ tương đương với độ phân giải vùng quét $655\text{ px}$. Khi hệ thống thu thập ảnh siêu âm từ các thiết bị chuẩn đoán hình ảnh khác, hoặc thay đổi độ phân giải ảnh xuất ra, số liệu đo khoảng cách tổn thương sẽ sai lệch nghiêm trọng nếu hằng số này không được hiệu chuẩn lại thông qua ma trận nội quan của máy quét mới.
|
Hằng số quy đổi $\text{PIXELTOMM} = \frac{45.0}{655.0}$ là một giá trị được cấu hình cứng (Hardcoded) trong mã nguồn, đặc trưng duy nhất cho một dòng máy siêu âm lâm sàng có tỷ lệ hiển thị $45mm$ tương đương với độ phân giải vùng quét $655\text{ px}$. Khi hệ thống thu thập ảnh siêu âm từ các thiết bị chuẩn đoán hình ảnh khác, hoặc thay đổi độ phân giải ảnh xuất ra, số liệu đo khoảng cách tổn thương sẽ sai lệch nghiêm trọng nếu hằng số này không được hiệu chuẩn lại thông qua ma trận nội quan của máy quét mới.
|
||||||
|
|
||||||
### 5.3 Quy tắc ánh xạ phân lớp (Class Remapping Matrix) đối với mô hình Custom
|
### 5.3 Quy tắc ánh xạ phân lớp (Class Remapping Matrix) đối với mô hình Custom
|
||||||
|
|
||||||
Hai mô hình tùy biến sâu phục vụ mặt cắt góc nhìn phía trên bánh chè (`UNet3+` và `EfficientFeedback Network`) được huấn luyện trên tập dữ liệu đặc thù sở hữu thứ tự cấu trúc mảng nhãn đầu ra lệch pha hoàn toàn so với kiến trúc phân cấp chuẩn của hệ thống. Để thống nhất dữ liệu trả về cho Client, khối Backend API thực hiện cơ chế tự động chuyển đổi chỉ mục mảng (Index Remapping) theo bảng đặc tả logic dưới đây:
|
Hai mô hình tùy biến sâu phục vụ mặt cắt góc nhìn phía trên bánh chè (`UNet3+` và `EfficientFeedback Network`) được huấn luyện trên tập dữ liệu đặc thù sở hữu thứ tự cấu trúc mảng nhãn đầu ra lệch pha hoàn toàn so với kiến trúc phân cấp chuẩn của hệ thống. Để thống nhất dữ liệu trả về cho Client, khối Backend API thực hiện cơ chế tự động chuyển đổi chỉ mục mảng (Index Remapping) theo bảng đặc tả logic dưới đây:
|
||||||
|
|
||||||
|
|
||||||
| Chỉ mục Mô hình gốc (Output Model Index) | Chỉ mục chuẩn hóa hệ thống (Standard System Index) | Tên nhãn lớp giải phẫu tương ứng (Anatomical Label Class) |
|
| Chỉ mục Mô hình gốc (Output Model Index) | Chỉ mục chuẩn hóa hệ thống (Standard System Index) | Tên nhãn lớp giải phẫu tương ứng (Anatomical Label Class) |
|
||||||
| --- | --- | --- |
|
| ---------------------------------------- | -------------------------------------------------- | --------------------------------------------------------- |
|
||||||
| `0` | `0` | **`background`** (Nền ảnh không chứa cấu trúc) |
|
| `0` | `0` | `background` (Nền ảnh không chứa cấu trúc) |
|
||||||
| `1` | `2` | **`fat`** (Lớp mô mỡ dưới da) |
|
| `1` | `2` | `fat` (Lớp mô mỡ dưới da) |
|
||||||
| `2` | `6` | **`tendon`** (Cấu trúc gân cơ) |
|
| `2` | `6` | `tendon` (Cấu trúc gân cơ) |
|
||||||
| `3` | `1` | **`effusion`** (Vùng tụ dịch khớp gối ổ viêm) |
|
| `3` | `1` | `effusion` (Vùng tụ dịch khớp gối ổ viêm) |
|
||||||
| `4` | `4` | **`femur`** (Ranh giới cấu trúc xương đùi) |
|
| `4` | `4` | `femur` (Ranh giới cấu trúc xương đùi) |
|
||||||
| `5` | `5` | **`synovium`** (Màng hoạt dịch bao quanh khớp) |
|
| `5` | `5` | `synovium` (Màng hoạt dịch bao quanh khớp) |
|
||||||
| `6` | `3` | **`fat-pat`** (Tổ chức mỡ Hoffa) |
|
| `6` | `3` | `fat-pat` (Tổ chức mỡ Hoffa) |
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 5.4 Cơ chế tự động dọn dẹp tập tin tồn đọng (Garbage Collection Task)
|
### 5.4 Cơ chế tự động dọn dẹp tập tin tồn đọng (Garbage Collection Task)
|
||||||
|
|
||||||
@@ -388,8 +423,12 @@ Các tập tin ảnh thô tải lên thư mục `uploads/` và ảnh xử lý nh
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 6. Giải pháp mở rộng tính năng mã nguồn (Backend Optimization Guide)
|
## 6. Giải pháp mở rộng tính năng mã nguồn (Backend Optimization Guide)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 6.1 Tăng tốc độ phản hồi bằng Cơ chế Caching Mô hình Toàn cục
|
### 6.1 Tăng tốc độ phản hồi bằng Cơ chế Caching Mô hình Toàn cục
|
||||||
|
|
||||||
Thay thế kiến trúc nạp tải mô hình cũ bằng một kho lưu trữ Cache tĩnh trong bộ nhớ RAM, tối ưu hóa thời gian xử lý request từ mức giây xuống mức mili-giây:
|
Thay thế kiến trúc nạp tải mô hình cũ bằng một kho lưu trữ Cache tĩnh trong bộ nhớ RAM, tối ưu hóa thời gian xử lý request từ mức giây xuống mức mili-giây:
|
||||||
@@ -409,11 +448,13 @@ def get_cached_angle_model(selected_model_name: str):
|
|||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 6.2 Thêm mới một kiến trúc phân loại góc chụp (Ví dụ: Vision Transformer - ViT)
|
### 6.2 Thêm mới một kiến trúc phân loại góc chụp (Ví dụ: Vision Transformer - ViT)
|
||||||
|
|
||||||
Để tích hợp một mạng nơ-ron mới vào hệ thống xử lý, tuân thủ nghiêm ngặt quy trình 3 bước sau:
|
Để tích hợp một mạng nơ-ron mới vào hệ thống xử lý, tuân thủ nghiêm ngặt quy trình 3 bước sau:
|
||||||
|
|
||||||
* **Bước 1:** Bổ sung khối xử lý điều kiện rẽ nhánh logic vào hàm khởi tạo mô hình `load_angle_model()`:
|
- **Bước 1:** Bổ sung khối xử lý điều kiện rẽ nhánh logic vào hàm khởi tạo mô hình `load_angle_model()`:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
elif model_name == 'vit':
|
elif model_name == 'vit':
|
||||||
@@ -428,9 +469,8 @@ elif model_name == 'vit':
|
|||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
* **Bước 2:** Di chuyển tệp trọng số huấn luyện nhị phân của mạng (`best_vit_b16.pth`) vào chính xác không gian lưu trữ của thư mục `/models/`.
|
- **Bước 2:** Di chuyển tệp trọng số huấn luyện nhị phân của mạng (`best_vit_b16.pth`) vào chính xác không gian lưu trữ của thư mục `/models/`.
|
||||||
|
- **Bước 3:** Ứng dụng phía Client có thể kích hoạt mạng mới bằng cách truyền giá trị định danh qua tham số URL: `/api/analyze?angle_model=vit`.
|
||||||
* **Bước 3:** Ứng dụng phía Client có thể kích hoạt mạng mới bằng cách truyền giá trị định danh qua tham số URL: `/api/analyze?angle_model=vit`.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -460,6 +500,8 @@ async def analyze_batch_images(images: List[UploadFile] = File(...)):
|
|||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 6.4 Bản đóng gói container hóa ứng dụng (Production Dockerfile)
|
### 6.4 Bản đóng gói container hóa ứng dụng (Production Dockerfile)
|
||||||
|
|
||||||
Đóng gói toàn bộ ML Stack bao gồm trình điều khiển GPU NVIDIA CUDA để triển khai đồng bộ trên các hạ tầng Cloud hoặc máy chủ On-Premise của bệnh viện:
|
Đóng gói toàn bộ ML Stack bao gồm trình điều khiển GPU NVIDIA CUDA để triển khai đồng bộ trên các hạ tầng Cloud hoặc máy chủ On-Premise của bệnh viện:
|
||||||
@@ -490,15 +532,16 @@ CMD ["python", "app.py"]
|
|||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
* **Lệnh khởi dựng Image hệ thống:** `docker build -t medical-api-service .`
|
- **Lệnh khởi dựng Image hệ thống:** `docker build -t medical-api-service .`
|
||||||
|
- **Lệnh kích hoạt Container chia sẻ tài nguyên phần cứng GPU vật lý:**
|
||||||
* **Lệnh kích hoạt Container chia sẻ tài nguyên phần cứng GPU vật lý:**
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker run --gpus all -p 8000:8000 -v $(pwd)/models:/app/models medical-api-service
|
docker run --gpus all -p 8000:8000 -v $(pwd)/models:/app/models medical-api-service
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 6.5 Bộ chuyển đổi tiếp nhận trực tiếp luồng dữ liệu ảnh y tế chuẩn DICOM
|
### 6.5 Bộ chuyển đổi tiếp nhận trực tiếp luồng dữ liệu ảnh y tế chuẩn DICOM
|
||||||
|
|
||||||
Mở rộng chức năng cho phép hệ thống API đọc trực tiếp tệp tin ảnh gốc dạng `.dcm` trích xuất trực tiếp từ các thiết bị siêu âm chuẩn lâm sàng trong bệnh viện mà không cần qua bước chuyển đổi định dạng thủ công:
|
Mở rộng chức năng cho phép hệ thống API đọc trực tiếp tệp tin ảnh gốc dạng `.dcm` trích xuất trực tiếp từ các thiết bị siêu âm chuẩn lâm sàng trong bệnh viện mà không cần qua bước chuyển đổi định dạng thủ công:
|
||||||
@@ -527,16 +570,21 @@ async def analyze_dicom_file(file: UploadFile = File(...)):
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## Phụ lục: Đặc tả Dữ liệu định lượng lâm sàng
|
## Phụ lục: Đặc tả Dữ liệu định lượng lâm sàng
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### Phụ lục A: Bảng phân định mã màu mặt nạ phân đoạn ngữ nghĩa (Color Map Legend)
|
### Phụ lục A: Bảng phân định mã màu mặt nạ phân đoạn ngữ nghĩa (Color Map Legend)
|
||||||
|
|
||||||
1. Cấu trúc Mặt cắt mặt trên bánh chè - Góc SUP (`sup-up-long`)
|
1. Cấu trúc Mặt cắt mặt trên bánh chè - Góc SUP (`sup-up-long`)
|
||||||
|
|
||||||
Góc SUP tập trung khoanh vùng các lớp mô mềm phía trước đầu gối phục vụ thuật toán tính toán độ dày dịch tụ.
|
Góc SUP tập trung khoanh vùng các lớp mô mềm phía trước đầu gối phục vụ thuật toán tính toán độ dày dịch tụ.
|
||||||
|
|
||||||
|
|
||||||
| Từ khóa nhãn (Key) | Cấu trúc giải phẫu đích | Mã màu hiển thị (RGB) | Trực quan hóa màu sắc |
|
| Từ khóa nhãn (Key) | Cấu trúc giải phẫu đích | Mã màu hiển thị (RGB) | Trực quan hóa màu sắc |
|
||||||
| --- | --- | --- | --- |
|
| ------------------ | ---------------------------- | --------------------- | -------------------------- |
|
||||||
| `background` | Nền ảnh siêu âm | `[0, 0, 0]` | ⬛ Đen (Không chứa dữ liệu) |
|
| `background` | Nền ảnh siêu âm | `[0, 0, 0]` | ⬛ Đen (Không chứa dữ liệu) |
|
||||||
| `effusion` | Vùng dịch khớp tụ ổ viêm | `[255, 0, 0]` | 🟥 Đỏ |
|
| `effusion` | Vùng dịch khớp tụ ổ viêm | `[255, 0, 0]` | 🟥 Đỏ |
|
||||||
| `fat` | Tổ chức mô mỡ dưới da | `[255, 255, 0]` | 🟨 Vàng |
|
| `fat` | Tổ chức mô mỡ dưới da | `[255, 255, 0]` | 🟨 Vàng |
|
||||||
@@ -545,15 +593,15 @@ Góc SUP tập trung khoanh vùng các lớp mô mềm phía trước đầu g
|
|||||||
| `synovium` | Lớp màng hoạt dịch tăng sinh | `[255, 0, 255]` | 🟪 Tím |
|
| `synovium` | Lớp màng hoạt dịch tăng sinh | `[255, 0, 255]` | 🟪 Tím |
|
||||||
| `tendon` | Vùng bó gân cơ | `[0, 0, 255]` | 🟦 Xanh dương |
|
| `tendon` | Vùng bó gân cơ | `[0, 0, 255]` | 🟦 Xanh dương |
|
||||||
|
|
||||||
> 🔄 **QUY TẮC CHUYỂN ĐỔI CHUYỂN GÓC (SUP $\rightarrow$ POST):**
|
|
||||||
> Khi hệ thống chuyển đổi trạng thái phân tích sang mặt cắt phía sau khớp gối (Góc `POST`), ma trận thuật toán phân đoạn sẽ tự động tái cấu trúc màu sắc ngữ nghĩa: Vùng tổn thương chứa **`effusion`** (màu đỏ) sẽ chuyển trạng thái biểu diễn thành **`baker's cyst`** (Kén Baker), và tổ chức cấu trúc vùng **`fat-pat`** (màu lam sáng) sẽ hoán đổi ý nghĩa thành vùng **`muscle`** (Cơ bắp vùng khoeo).
|
|
||||||
>
|
|
||||||
>
|
|
||||||
|
|
||||||
2. Cấu trúc Mặt cắt mặt sau vùng khoeo chân - Góc POST (`post-trans`)
|
> 🔄 **QUY TẮC CHUYỂN ĐỔI CHUYỂN GÓC (SUP $\rightarrow$ POST):**
|
||||||
|
> Khi hệ thống chuyển đổi trạng thái phân tích sang mặt cắt phía sau khớp gối (Góc `POST`), ma trận thuật toán phân đoạn sẽ tự động tái cấu trúc màu sắc ngữ nghĩa: Vùng tổn thương chứa `effusion` (màu đỏ) sẽ chuyển trạng thái biểu diễn thành `baker's cyst` (Kén Baker), và tổ chức cấu trúc vùng `fat-pat` (màu lam sáng) sẽ hoán đổi ý nghĩa thành vùng `muscle` (Cơ bắp vùng khoeo).
|
||||||
|
|
||||||
|
1. Cấu trúc Mặt cắt mặt sau vùng khoeo chân - Góc POST (`post-trans`)
|
||||||
|
|
||||||
|
|
||||||
| Từ khóa nhãn (Key) | Cấu trúc giải phẫu đích | Mã màu hiển thị (RGB) | Trực quan hóa màu sắc |
|
| Từ khóa nhãn (Key) | Cấu trúc giải phẫu đích | Mã màu hiển thị (RGB) | Trực quan hóa màu sắc |
|
||||||
| --- | --- | --- | --- |
|
| ------------------ | ---------------------------------------- | --------------------- | --------------------- |
|
||||||
| `background` | Nền ảnh siêu âm | `[0, 0, 0]` | ⬛ Đen |
|
| `background` | Nền ảnh siêu âm | `[0, 0, 0]` | ⬛ Đen |
|
||||||
| `baker's cyst` | Tổ chức kén hoạt dịch vùng khoeo (Baker) | `[255, 0, 0]` | 🟥 Đỏ |
|
| `baker's cyst` | Tổ chức kén hoạt dịch vùng khoeo (Baker) | `[255, 0, 0]` | 🟥 Đỏ |
|
||||||
| `fat` | Lớp mô mỡ | `[255, 255, 0]` | 🟨 Vàng |
|
| `fat` | Lớp mô mỡ | `[255, 255, 0]` | 🟨 Vàng |
|
||||||
@@ -562,17 +610,21 @@ Góc SUP tập trung khoanh vùng các lớp mô mềm phía trước đầu g
|
|||||||
| `synovium` | Màng hoạt dịch mặt sau | `[255, 0, 255]` | 🟪 Tím |
|
| `synovium` | Màng hoạt dịch mặt sau | `[255, 0, 255]` | 🟪 Tím |
|
||||||
| `tendon` | Hệ thống gân cơ mặt sau | `[0, 0, 255]` | 🟦 Xanh dương |
|
| `tendon` | Hệ thống gân cơ mặt sau | `[0, 0, 255]` | 🟦 Xanh dương |
|
||||||
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### Phụ lục B: Thang điểm đánh giá mức độ nghiêm trọng của ổ viêm (Clinical Severity Score)
|
### Phụ lục B: Thang điểm đánh giá mức độ nghiêm trọng của ổ viêm (Clinical Severity Score)
|
||||||
|
|
||||||
Hệ thống chấm điểm toán học tự động căn cứ trên trọng số diện tích và độ dày phân tách để đưa ra kết luận mức độ bệnh lý lâm sàng thông qua phương trình tuyến tính tổng hợp:
|
Hệ thống chấm điểm toán học tự động căn cứ trên trọng số diện tích và độ dày phân tách để đưa ra kết luận mức độ bệnh lý lâm sàng thông qua phương trình tuyến tính tổng hợp:
|
||||||
|
|
||||||
$$\text{combined\_score} = \text{effusion\_score} \times 0.6 + \text{synovium\_score} \times 0.4$$
|
$$\text{combinedscore} = \text{effusionscore} \times 0.6 + \text{synoviumscore} \times 0.4$$
|
||||||
|
|
||||||
Dựa trên kết quả giá trị của biến số $\text{combined\_score}$, hệ thống tự động phân cấp thành 4 ngưỡng trạng thái lâm sàng tương ứng:
|
Dựa trên kết quả giá trị của biến số $\text{combinedscore}$, hệ thống tự động phân cấp thành 4 ngưỡng trạng thái lâm sàng tương ứng:
|
||||||
|
|
||||||
|
- **Mức 0 - Rất nhẹ ($\text{score} < 3$):** Trạng thái ổ dịch khớp và cấu trúc màng hoạt dịch nằm hoàn toàn trong giới hạn sinh lý bình thường của cơ thể.
|
||||||
|
- **Mức 1 - Nhẹ ($\text{score}$ từ $3$ đến $7.9$):** Xuất hiện hiện tượng tụ dịch khớp lớp mỏng, màng hoạt dịch có dấu hiệu tăng sinh nhẹ cấu trúc màng.
|
||||||
|
- **Mức 2 - Trung bình ($\text{score}$ từ $8$ đến $15$):** Lượng dịch tụ khớp gối ở mức độ vừa phải, màng hoạt dịch bắt đầu phì đại và tăng sinh rõ nét.
|
||||||
|
- **Mức 3 - Nặng ($\text{score} > 15$):** Lớp tụ dịch khớp gối dày kích thước lớn, màng hoạt dịch tăng sinh phì đại mạnh, lan rộng diện tích cấu trúc giải phẫu xung quanh.
|
||||||
|
|
||||||
* **Mức 0 - Rất nhẹ ($\text{score} < 3$):** Trạng thái ổ dịch khớp và cấu trúc màng hoạt dịch nằm hoàn toàn trong giới hạn sinh lý bình thường của cơ thể.
|
|
||||||
* **Mức 1 - Nhẹ ($\text{score}$ từ $3$ đến $7.9$):** Xuất hiện hiện tượng tụ dịch khớp lớp mỏng, màng hoạt dịch có dấu hiệu tăng sinh nhẹ cấu trúc màng.
|
|
||||||
* **Mức 2 - Trung bình ($\text{score}$ từ $8$ đến $15$):** Lượng dịch tụ khớp gối ở mức độ vừa phải, màng hoạt dịch bắt đầu phì đại và tăng sinh rõ nét.
|
|
||||||
* **Mức 3 - Nặng ($\text{score} > 15$):** Lớp tụ dịch khớp gối dày kích thước lớn, màng hoạt dịch tăng sinh phì đại mạnh, lan rộng diện tích cấu trúc giải phẫu xung quanh.
|
|
||||||
233
workspace/sprint_1_2/CODEBASE/backend/api/analysis_api.py
Normal file
233
workspace/sprint_1_2/CODEBASE/backend/api/analysis_api.py
Normal file
@@ -0,0 +1,233 @@
|
|||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, status, UploadFile, File
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
from datetime import datetime
|
||||||
|
from data.spec.schemas import (
|
||||||
|
AnalysisJobSubmit, JobStatus, PipelineStep, StepEvent,
|
||||||
|
ModelCatalog, ModelRegistrationResult, HealthStatus,
|
||||||
|
AnalysisJobSyncSubmit, JobResult, ErrorResponse,
|
||||||
|
)
|
||||||
|
from backend.implementation.analysis_jobs import service as analysis_service
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["analysis"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
_event_queues: dict[str, asyncio.Queue] = {}
|
||||||
|
_queue_lock = asyncio.Lock()
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def _get_queue(job_id: str):
|
||||||
|
async with _queue_lock:
|
||||||
|
if job_id not in _event_queues:
|
||||||
|
_event_queues[job_id] = asyncio.Queue()
|
||||||
|
yield _event_queues[job_id]
|
||||||
|
|
||||||
|
|
||||||
|
def _get_queue_sync(job_id: str) -> asyncio.Queue:
|
||||||
|
if job_id not in _event_queues:
|
||||||
|
_event_queues[job_id] = asyncio.Queue()
|
||||||
|
return _event_queues[job_id]
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _sse_format(event: StepEvent) -> str:
|
||||||
|
lines = [f"event: {event.event_type}"]
|
||||||
|
payload = event.model_dump(mode="json")
|
||||||
|
lines.append(f"data: {payload}")
|
||||||
|
lines.append("")
|
||||||
|
lines.append("")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/analysis-jobs",
|
||||||
|
response_model=dict,
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def submit_analysis_job(
|
||||||
|
payload: AnalysisJobSubmit,
|
||||||
|
user_id: str = Depends(_verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
job_id = await analysis_service.submit_job(
|
||||||
|
session_id=payload.session_id,
|
||||||
|
params=payload.params or {},
|
||||||
|
model_versions=payload.model_versions,
|
||||||
|
)
|
||||||
|
return {"job_id": job_id}
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/analysis-jobs/{job_id}",
|
||||||
|
response_model=JobStatus,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def get_job_status(job_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await analysis_service.job_status(job_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/analysis-jobs/{job_id}/steps",
|
||||||
|
response_model=list[PipelineStep],
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def get_job_steps(job_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await analysis_service.job_steps(job_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/analysis",
|
||||||
|
response_model=JobResult,
|
||||||
|
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def submit_sync_analysis(
|
||||||
|
payload: AnalysisJobSyncSubmit,
|
||||||
|
user_id: str = Depends(_verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await analysis_service.submit_sync(
|
||||||
|
session_id=payload.session_id,
|
||||||
|
params=payload.params or {},
|
||||||
|
model_versions=payload.model_versions,
|
||||||
|
)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/analysis-jobs/{job_id}/stream",
|
||||||
|
responses={
|
||||||
|
401: {"model": ErrorResponse},
|
||||||
|
404: {"model": ErrorResponse},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
async def stream_job_events(job_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
queue = _get_queue_sync(job_id)
|
||||||
|
|
||||||
|
async def event_generator():
|
||||||
|
try:
|
||||||
|
while True:
|
||||||
|
event = await queue.get()
|
||||||
|
yield _sse_format(event)
|
||||||
|
if event.event_type == "completed" or event.event_type == "failed":
|
||||||
|
break
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
logger.info(f"SSE stream cancelled for job_id={job_id}")
|
||||||
|
finally:
|
||||||
|
async with _queue_lock:
|
||||||
|
_event_queues.pop(job_id, None)
|
||||||
|
|
||||||
|
return StreamingResponse(
|
||||||
|
event_generator(),
|
||||||
|
media_type="text/event-stream",
|
||||||
|
headers={
|
||||||
|
"Cache-Control": "no-cache",
|
||||||
|
"X-Accel-Buffering": "no",
|
||||||
|
"Connection": "keep-alive",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/internal/analysis-jobs/{job_id}/events",
|
||||||
|
status_code=status.HTTP_202_ACCEPTED,
|
||||||
|
include_in_schema=False,
|
||||||
|
)
|
||||||
|
async def internal_push_event(job_id: str, event: dict):
|
||||||
|
queue = _get_queue_sync(job_id)
|
||||||
|
try:
|
||||||
|
step_event = StepEvent(
|
||||||
|
step_id=event.get("step_id", ""),
|
||||||
|
job_id=job_id,
|
||||||
|
event_type=event.get("event_type", "progress"),
|
||||||
|
task_type=event.get("task_type", ""),
|
||||||
|
status=event.get("status", "running"),
|
||||||
|
data=event.get("data"),
|
||||||
|
timestamp=datetime.now(),
|
||||||
|
)
|
||||||
|
await queue.put(step_event)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.error(f"Failed to push event for job {job_id}: {exc}")
|
||||||
|
return {"queued": True}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/health",
|
||||||
|
response_model=HealthStatus,
|
||||||
|
include_in_schema=False,
|
||||||
|
)
|
||||||
|
async def health_check():
|
||||||
|
try:
|
||||||
|
return await analysis_service.health()
|
||||||
|
except NotImplementedError:
|
||||||
|
return HealthStatus(
|
||||||
|
status="ok",
|
||||||
|
version="0.1.0",
|
||||||
|
dependencies={},
|
||||||
|
uptime_seconds=0.0,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/model-registry",
|
||||||
|
response_model=ModelCatalog,
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def list_models(user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await analysis_service.list_registered_models()
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/models/register",
|
||||||
|
response_model=ModelRegistrationResult,
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def register_model(
|
||||||
|
model_id: str,
|
||||||
|
file: UploadFile | None = None,
|
||||||
|
user_id: str = Depends(_verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await analysis_service.register_model(model_id, file)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
82
workspace/sprint_1_2/CODEBASE/backend/api/auth_api.py
Normal file
82
workspace/sprint_1_2/CODEBASE/backend/api/auth_api.py
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from data.spec.schemas import LoginRequest, Token, UserProfile, UserUpdateRequest, RefreshRequest, ErrorResponse
|
||||||
|
from backend.implementation.auth import service as auth_service
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["auth"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
|
||||||
|
async def verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
profile = await auth_service.me(token)
|
||||||
|
return profile.user_id
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/auth/login",
|
||||||
|
response_model=Token,
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def login(payload: LoginRequest):
|
||||||
|
try:
|
||||||
|
return await auth_service.login(payload.username, payload.password)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/auth/logout",
|
||||||
|
status_code=status.HTTP_204_NO_CONTENT,
|
||||||
|
)
|
||||||
|
async def logout(token: str = Depends(oauth2_scheme)):
|
||||||
|
try:
|
||||||
|
await auth_service.logout(token)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/auth/refresh",
|
||||||
|
response_model=Token,
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def refresh(payload: RefreshRequest):
|
||||||
|
try:
|
||||||
|
return await auth_service.refresh(payload.refresh_token)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/users/me",
|
||||||
|
response_model=UserProfile,
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def get_me(user_id: str = Depends(verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await auth_service.me(user_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch(
|
||||||
|
"/users/me",
|
||||||
|
response_model=UserProfile,
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def update_me(payload: UserUpdateRequest, user_id: str = Depends(verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await auth_service.update_me(user_id, payload.model_dump(exclude_none=True))
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
47
workspace/sprint_1_2/CODEBASE/backend/api/ingestion_api.py
Normal file
47
workspace/sprint_1_2/CODEBASE/backend/api/ingestion_api.py
Normal file
@@ -0,0 +1,47 @@
|
|||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from data.spec.schemas import IngestionRecord, RecordDetail, ErrorResponse
|
||||||
|
from backend.implementation.ingestion_history import service as ingestion_service
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["ingestion-history"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/ingestion-history",
|
||||||
|
response_model=list[IngestionRecord],
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def list_ingestion_records(user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await ingestion_service.list_records(user_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/ingestion-history/{record_id}",
|
||||||
|
response_model=RecordDetail,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def get_ingestion_record(record_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await ingestion_service.get_record(record_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from data.spec.schemas import NotificationItem, NotificationPreferences, ErrorResponse
|
||||||
|
from backend.implementation.notification import service as notification_service
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["notification"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/notifications",
|
||||||
|
response_model=list[NotificationItem],
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def list_notifications(user_id: str = Depends(_verify_jwt_token), filters: dict | None = None):
|
||||||
|
try:
|
||||||
|
return await notification_service.list_notifications(user_id, filters)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch(
|
||||||
|
"/notifications/{notification_id}/read",
|
||||||
|
status_code=status.HTTP_204_NO_CONTENT,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def mark_read(notification_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
await notification_service.mark_read(notification_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/notifications/preferences",
|
||||||
|
status_code=status.HTTP_204_NO_CONTENT,
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def set_preferences(payload: NotificationPreferences, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
await notification_service.set_preferences(user_id, payload.model_dump(exclude_none=True))
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
91
workspace/sprint_1_2/CODEBASE/backend/api/patient_api.py
Normal file
91
workspace/sprint_1_2/CODEBASE/backend/api/patient_api.py
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from data.spec.schemas import Patient, PatientCreate, PatientListResponse, ErrorResponse
|
||||||
|
from backend.implementation.patient import service as patient_service
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["patient"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/patients",
|
||||||
|
response_model=PatientListResponse,
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def list_patients(user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
items = await patient_service.list_patients(user_id)
|
||||||
|
return PatientListResponse(items=items, total=len(items))
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/patients",
|
||||||
|
response_model=Patient,
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def create_patient(payload: PatientCreate, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await patient_service.create_patient(payload.model_dump(exclude_none=True))
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/patients/{patient_id}",
|
||||||
|
response_model=Patient,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def get_patient(patient_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await patient_service.get_patient(patient_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/patients/{patient_id}/sessions",
|
||||||
|
response_model=list[dict],
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def list_patient_sessions(patient_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await patient_service.list_sessions(patient_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/patients/{patient_id}/history",
|
||||||
|
response_model=list[dict],
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def patient_ingestion_history(patient_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await patient_service.ingestion_history(patient_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
320
workspace/sprint_1_2/CODEBASE/backend/api/safety_api.py
Normal file
320
workspace/sprint_1_2/CODEBASE/backend/api/safety_api.py
Normal file
@@ -0,0 +1,320 @@
|
|||||||
|
import asyncio
|
||||||
|
from typing import Any
|
||||||
|
import httpx
|
||||||
|
import logging
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, status, UploadFile, File
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
from PILOT_PROJECT.workspace.sprint_1_2.CODEBASE.data.spec.schemas.safety_schemas import ChatResponse
|
||||||
|
from data.spec.schemas import (
|
||||||
|
HeatmapResult, RationaleResult, ChatEvent, DriftCheckResult,
|
||||||
|
EvidenceList, ActivationMeta, AnnotationArtifact, EscalationTicket,
|
||||||
|
GuardrailResult, ErrorResponse, CorrectionSubmit, CorrectionRecord,
|
||||||
|
)
|
||||||
|
|
||||||
|
from backend.implementation.safety import service as safety_service
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["safety"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _sse_chat_format(event: ChatEvent) -> str:
|
||||||
|
lines = [f"event: {event.event_type}"]
|
||||||
|
payload = event.model_dump(mode="json")
|
||||||
|
lines.append(f"data: {payload}")
|
||||||
|
lines.append("")
|
||||||
|
lines.append("")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/explanations/gradcam",
|
||||||
|
response_model=HeatmapResult,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def gradcam(session_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await safety_service.gradcam(session_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/explanations/rationale",
|
||||||
|
response_model=RationaleResult,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def rationale(
|
||||||
|
session_id: str,
|
||||||
|
redaction_hash: str | None = None,
|
||||||
|
user_id: str = Depends(_verify_jwt_token)
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await safety_service.rationale(session_id, redaction_hash)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/safety/circuit-breaker",
|
||||||
|
status_code=status.HTTP_204_NO_CONTENT,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def circuit_breaker(session_id: str, payload: dict, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
await safety_service.circuit_break(session_id, payload.get("flag", False))
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/chat/socratic",
|
||||||
|
response_model=ChatResponse,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model: ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def socratic_chat(
|
||||||
|
session_id: str,
|
||||||
|
payload: dict,
|
||||||
|
redaction_hash: str | None = None,
|
||||||
|
user_id: str = Depends(_verify_jwt_token)
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await safety_service.socratic_chat(
|
||||||
|
session_id,
|
||||||
|
payload.get("prompt", ""),
|
||||||
|
redaction_hash=redaction_hash
|
||||||
|
)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/drift/check",
|
||||||
|
response_model=DriftCheckResult,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def drift_check(session_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await safety_service.drift_check(session_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/rag/evidence",
|
||||||
|
response_model=EvidenceList,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def rag_evidence(session_id: str, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await safety_service.rag_evidence(session_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/activations",
|
||||||
|
response_model=ActivationMeta,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def activations(session_id: str, params: dict, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await safety_service.activations(session_id, params)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/annotations/artifacts",
|
||||||
|
response_model=AnnotationArtifact,
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def upload_artifact(
|
||||||
|
session_id: str,
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
user_id: str = Depends(_verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await safety_service.upload_artifact(session_id, file)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/ground-truth",
|
||||||
|
status_code=status.HTTP_204_NO_CONTENT,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def ground_truth(session_id: str, payload: dict, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
await safety_service.ground_truth(session_id, payload.get("label", {}))
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/escalation",
|
||||||
|
response_model=EscalationTicket,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def escalate(session_id: str, payload: dict, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await safety_service.escalate(session_id, payload.get("reason", ""))
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/annotations/morphology",
|
||||||
|
status_code=status.HTTP_204_NO_CONTENT,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def morphology(session_id: str, payload: dict, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
await safety_service.morphology(session_id, payload.get("annotation", {}))
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/safety/guardrail-check",
|
||||||
|
response_model=GuardrailResult,
|
||||||
|
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def guardrail_check(payload: dict, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
session_id = payload.get("session_id", "")
|
||||||
|
return await safety_service.guardrail_check(
|
||||||
|
session_id=session_id,
|
||||||
|
prompt=payload.get("prompt", ""),
|
||||||
|
score=float(payload.get("score", 0.0)),
|
||||||
|
)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except (ValueError, TypeError) as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/feedback",
|
||||||
|
response_model=CorrectionRecord,
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
responses={
|
||||||
|
401: {"model": ErrorResponse},
|
||||||
|
404: {"model": ErrorResponse},
|
||||||
|
422: {"model": ErrorResponse},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
async def submit_correction(
|
||||||
|
session_id: str,
|
||||||
|
payload: CorrectionSubmit,
|
||||||
|
user_id: str = Depends(_verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await safety_service.submit_correction(session_id, payload.dict())
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/sessions/{session_id}/chat/stream",
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def chat_stream(
|
||||||
|
session_id: str,
|
||||||
|
prompt: str,
|
||||||
|
redaction_hash: str | None = None,
|
||||||
|
user_id: str = Depends(_verify_jwt_token),
|
||||||
|
):
|
||||||
|
async def generate():
|
||||||
|
try:
|
||||||
|
async for chunk in safety_service.chat_stream(session_id, prompt, redaction_hash):
|
||||||
|
event = ChatEvent(
|
||||||
|
session_id=session_id,
|
||||||
|
event_type="chunk",
|
||||||
|
content=chunk,
|
||||||
|
is_final=False,
|
||||||
|
)
|
||||||
|
yield _sse_chat_format(event)
|
||||||
|
final_event = ChatEvent(
|
||||||
|
session_id=session_id,
|
||||||
|
event_type="completed",
|
||||||
|
content="",
|
||||||
|
is_final=True,
|
||||||
|
)
|
||||||
|
yield _sse_chat_format(final_event)
|
||||||
|
except HTTPException as exc:
|
||||||
|
error_event = ChatEvent(
|
||||||
|
session_id=session_id,
|
||||||
|
event_type="error",
|
||||||
|
content=exc.detail,
|
||||||
|
is_final=True,
|
||||||
|
)
|
||||||
|
yield _sse_chat_format(error_event)
|
||||||
|
except NotImplementedError:
|
||||||
|
fallback_event = ChatEvent(
|
||||||
|
session_id=session_id,
|
||||||
|
event_type="error",
|
||||||
|
content="Chat streaming service not yet implemented",
|
||||||
|
is_final=True,
|
||||||
|
)
|
||||||
|
yield _sse_chat_format(fallback_event)
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
logger.info(f"Chat stream cancelled for session_id={session_id}")
|
||||||
|
|
||||||
|
return StreamingResponse(
|
||||||
|
generate(),
|
||||||
|
media_type="text/event-stream",
|
||||||
|
headers={
|
||||||
|
"Cache-Control": "no-cache",
|
||||||
|
"X-Accel-Buffering": "no",
|
||||||
|
"Connection": "keep-alive",
|
||||||
|
},
|
||||||
|
)
|
||||||
214
workspace/sprint_1_2/CODEBASE/backend/api/session_api.py
Normal file
214
workspace/sprint_1_2/CODEBASE/backend/api/session_api.py
Normal file
@@ -0,0 +1,214 @@
|
|||||||
|
from typing import Any
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, status, UploadFile, File
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from data.spec.schemas import (
|
||||||
|
Session, SessionDetail, SessionCreate, SessionPatchReview,
|
||||||
|
FrameMetadata, PersistResult, ExportResult, ScrubResult,
|
||||||
|
ReportCreate, ReportSignRequest, ReportSyncEMRRequest, SyncResult,
|
||||||
|
ErrorResponse,
|
||||||
|
)
|
||||||
|
from backend.implementation.session import service as session_service
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["session"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
|
||||||
|
async def verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions",
|
||||||
|
response_model=Session,
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def create_session(payload: SessionCreate, user_id: str = Depends(verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await session_service.create_session(
|
||||||
|
user_id=user_id,
|
||||||
|
patient_id=payload.patient_id,
|
||||||
|
case_id=getattr(payload, "case_id", None),
|
||||||
|
)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/sessions/{session_id}",
|
||||||
|
response_model=SessionDetail,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def get_session(session_id: str, user_id: str = Depends(verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await session_service.get_session(session_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/frames",
|
||||||
|
response_model=FrameMetadata,
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def add_frame(
|
||||||
|
session_id: str,
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
frame_number: int | None = None,
|
||||||
|
user_id: str = Depends(verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await session_service.add_frame(session_id, file, frame_number)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch(
|
||||||
|
"/sessions/{session_id}/review",
|
||||||
|
response_model=Session,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def patch_review(
|
||||||
|
session_id: str,
|
||||||
|
payload: SessionPatchReview,
|
||||||
|
user_id: str = Depends(verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await session_service.patch_review(session_id, payload.model_dump(exclude_none=True))
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/reports",
|
||||||
|
response_model=dict,
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def create_report(payload: ReportCreate, user_id: str = Depends(verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
result = await session_service.persist(payload.session_id, payload.payload)
|
||||||
|
return {"report_id": result.session_id, "status": result.status, "updated_at": result.updated_at.isoformat()}
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/reports/{report_id}/sign",
|
||||||
|
response_model=dict,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def sign_report(
|
||||||
|
report_id: str,
|
||||||
|
payload: ReportSignRequest,
|
||||||
|
user_id: str = Depends(verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
result = await session_service.persist(report_id, {"signed": True, "signature": payload.signature})
|
||||||
|
return {"report_id": report_id, "signed": True, "updated_at": result.updated_at.isoformat()}
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/reports/{report_id}/emr-sync",
|
||||||
|
response_model=SyncResult,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def sync_emr(
|
||||||
|
report_id: str,
|
||||||
|
payload: ReportSyncEMRRequest,
|
||||||
|
user_id: str = Depends(verify_jwt_token),
|
||||||
|
):
|
||||||
|
from datetime import datetime
|
||||||
|
try:
|
||||||
|
result = await session_service.persist(report_id, {"emr_sync": True})
|
||||||
|
return SyncResult(
|
||||||
|
report_id=report_id,
|
||||||
|
emr_status="pending",
|
||||||
|
emr_reference=None,
|
||||||
|
synced_at=datetime.utcnow(),
|
||||||
|
)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/persist",
|
||||||
|
response_model=PersistResult,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def persist_session(
|
||||||
|
session_id: str,
|
||||||
|
review: dict,
|
||||||
|
user_id: str = Depends(verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await session_service.persist(session_id, review)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/export-pdf",
|
||||||
|
response_model=ExportResult,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def export_pdf(
|
||||||
|
session_id: str,
|
||||||
|
params: dict,
|
||||||
|
user_id: str = Depends(verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await session_service.export_pdf(session_id, params)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/sessions/{session_id}/scrub-validate",
|
||||||
|
response_model=ScrubResult,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def scrub_validate(
|
||||||
|
session_id: str,
|
||||||
|
metadata: dict,
|
||||||
|
user_id: str = Depends(verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await session_service.scrub_validate(session_id, metadata)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
47
workspace/sprint_1_2/CODEBASE/backend/api/settings_api.py
Normal file
47
workspace/sprint_1_2/CODEBASE/backend/api/settings_api.py
Normal file
@@ -0,0 +1,47 @@
|
|||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from data.spec.schemas import UserSettings, SettingsUpdate, ErrorResponse
|
||||||
|
from backend.implementation.settings import service as settings_service
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["settings"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get(
|
||||||
|
"/settings",
|
||||||
|
response_model=UserSettings,
|
||||||
|
responses={401: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def get_settings(user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await settings_service.get_settings(user_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch(
|
||||||
|
"/settings",
|
||||||
|
response_model=UserSettings,
|
||||||
|
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def update_settings(payload: SettingsUpdate, user_id: str = Depends(_verify_jwt_token)):
|
||||||
|
try:
|
||||||
|
return await settings_service.update_settings(user_id, payload.model_dump(exclude_none=True))
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
40
workspace/sprint_1_2/CODEBASE/backend/api/telemetry_api.py
Normal file
40
workspace/sprint_1_2/CODEBASE/backend/api/telemetry_api.py
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from data.spec.schemas import AnomalyReport, AnomalyRecord, ErrorResponse
|
||||||
|
from backend.implementation.telemetry import service as telemetry_service
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["telemetry"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
|
||||||
|
async def verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/analysis-jobs/{job_id}/anomalies",
|
||||||
|
response_model=AnomalyRecord,
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def report_anomaly(
|
||||||
|
job_id: str,
|
||||||
|
payload: AnomalyReport,
|
||||||
|
user_id: str = Depends(verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await telemetry_service.report_anomaly(job_id, payload.data)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except LookupError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
|
||||||
93
workspace/sprint_1_2/CODEBASE/backend/cv_inference_server.py
Normal file
93
workspace/sprint_1_2/CODEBASE/backend/cv_inference_server.py
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
"""
|
||||||
|
Standalone FastAPI server for CV inference (Modal Triton).
|
||||||
|
|
||||||
|
Run from CODEBASE root:
|
||||||
|
|
||||||
|
PYTHONPATH=. python -m backend.cv_inference_server
|
||||||
|
|
||||||
|
Or the backward-compatible launcher:
|
||||||
|
|
||||||
|
PYTHONPATH=. python backend/tests/test_fast_api_proxy.py
|
||||||
|
|
||||||
|
Default: http://127.0.0.1:8001 — point the frontend Vite proxy here (see .env.development).
|
||||||
|
|
||||||
|
Env:
|
||||||
|
TRITON_ENDPOINT Modal Triton KServe v2 HTTP URL
|
||||||
|
CV_INFERENCE_HOST bind host (default 127.0.0.1)
|
||||||
|
CV_INFERENCE_PORT bind port (default 8001)
|
||||||
|
ANGLE_MODEL / INFLAMMATION_MODEL / SEGMENT_MODEL optional overrides
|
||||||
|
CV_PIPELINE_VERSION cache invalidation fingerprint (default poc-v2-spec-cv)
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
# Must run before backend imports — config reads TRITON_ENDPOINT at import time.
|
||||||
|
DEFAULT_TRITON_ENDPOINT = "https://dtj-tran--triton-s3-service-unified-triton-server.modal.run"
|
||||||
|
os.environ.setdefault("TRITON_ENDPOINT", DEFAULT_TRITON_ENDPOINT)
|
||||||
|
|
||||||
|
import uvicorn
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
|
||||||
|
from backend.routers import cv_inference
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
DEFAULT_TRITON_ENDPOINT = "https://dtj-tran--triton-s3-service-unified-triton-server.modal.run"
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(app: FastAPI):
|
||||||
|
logger.info("Starting CV inference service on Triton: %s", os.getenv("TRITON_ENDPOINT"))
|
||||||
|
from backend.services.triton_warmup import warmup_triton_models
|
||||||
|
|
||||||
|
try:
|
||||||
|
await warmup_triton_models()
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("Triton warmup failed — first clinical request may be slow: %s", exc)
|
||||||
|
yield
|
||||||
|
logger.info("Shutting down CV inference service")
|
||||||
|
|
||||||
|
|
||||||
|
def create_app() -> FastAPI:
|
||||||
|
app = FastAPI(
|
||||||
|
title="VKIST CV Inference Service",
|
||||||
|
version="0.2.0",
|
||||||
|
description=(
|
||||||
|
"Spec-compliant musculoskeletal ultrasound CV pipeline "
|
||||||
|
"(CLAHE → angle → inflammation → conditional segmentation)."
|
||||||
|
),
|
||||||
|
lifespan=lifespan,
|
||||||
|
)
|
||||||
|
|
||||||
|
app.add_middleware(
|
||||||
|
CORSMiddleware,
|
||||||
|
allow_origins=os.getenv(
|
||||||
|
"CORS_ORIGINS",
|
||||||
|
"http://localhost:3000,http://localhost:5173,http://localhost:4173,http://127.0.0.1:5173",
|
||||||
|
).split(","),
|
||||||
|
allow_credentials=True,
|
||||||
|
allow_methods=["*"],
|
||||||
|
allow_headers=["*"],
|
||||||
|
)
|
||||||
|
|
||||||
|
app.include_router(cv_inference.router)
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
app = create_app()
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
logging.basicConfig(level=logging.INFO)
|
||||||
|
host = os.getenv("CV_INFERENCE_HOST", os.getenv("SEGMENT_TEST_HOST", "127.0.0.1"))
|
||||||
|
port = int(os.getenv("CV_INFERENCE_PORT", os.getenv("SEGMENT_TEST_PORT", "8001")))
|
||||||
|
logger.info("CV inference service listening on %s:%s", host, port)
|
||||||
|
uvicorn.run(app, host=host, port=port, log_level="info")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import logging
|
||||||
|
from typing import NamedTuple, Optional
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class DriftResult:
|
||||||
|
score: float
|
||||||
|
is_drifted: bool
|
||||||
|
threshold: float
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class GuardrailResult:
|
||||||
|
verdict: str # "PASS" | "MITIGATE"
|
||||||
|
reason: Optional[str] = None
|
||||||
|
|
||||||
|
class BERTAdapter:
|
||||||
|
"""
|
||||||
|
Adapter for BERT-based safety checks (drift, referee, guardrails).
|
||||||
|
Current implementation provides stubs.
|
||||||
|
"""
|
||||||
|
def __init__(self):
|
||||||
|
self.drift_threshold = 0.7
|
||||||
|
|
||||||
|
def drift_check(self, text: str) -> DriftResult:
|
||||||
|
"""
|
||||||
|
Checks if the input text drifts from the clinical domain.
|
||||||
|
"""
|
||||||
|
# Stub: Always return no drift
|
||||||
|
return DriftResult(score=0.9, is_drifted=False, threshold=self.drift_threshold)
|
||||||
|
|
||||||
|
def referee_check(self, text: str, retrieved_chunks: list) -> bool:
|
||||||
|
"""
|
||||||
|
RAG-Referee: Validates if LLM response is grounded in provided chunks.
|
||||||
|
"""
|
||||||
|
# Stub: Always return grounded
|
||||||
|
return True
|
||||||
|
|
||||||
|
def guardrail_check(self, text: str) -> GuardrailResult:
|
||||||
|
"""
|
||||||
|
Token/Chunk level guardrail check for hallucinations or scope violations.
|
||||||
|
"""
|
||||||
|
# Stub: Always return PASS
|
||||||
|
return GuardrailResult(verdict="PASS")
|
||||||
|
|
||||||
|
def get_bert_adapter() -> BERTAdapter:
|
||||||
|
return BERTAdapter()
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
import logging
|
||||||
|
from typing import Any, AsyncGenerator, List, Optional
|
||||||
|
from langchain_google_vertexai import VertexAI
|
||||||
|
from langchain.callbacks.base import BaseCallbackHandler
|
||||||
|
from langchain.schema import LLMResult
|
||||||
|
from backend.implementation import config
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class AuditCallbackHandler(BaseCallbackHandler):
|
||||||
|
"""
|
||||||
|
Langchain callback to enforce audit logging before LLM calls per NFR-16a.
|
||||||
|
"""
|
||||||
|
def __init__(self, session_id: str, metadata: Optional[dict] = None):
|
||||||
|
self.session_id = session_id
|
||||||
|
self.metadata = metadata or {}
|
||||||
|
|
||||||
|
def on_llm_start(self, serialized: Any, prompts: List[str], **kwargs) -> None:
|
||||||
|
# MANDATORY: Write egress_consent + egress_redact_manifest to immutable audit log
|
||||||
|
# In a real implementation, this would call a database service to commit to Postgres.
|
||||||
|
logger.info(f"[AUDIT] Pre-egress audit commit for session {self.session_id}. "
|
||||||
|
f"Prompts: {prompts}. Metadata: {self.metadata}")
|
||||||
|
|
||||||
|
def on_llm_end(self, response: LLMResult, **kwargs) -> None:
|
||||||
|
# Log actual egress event after completion
|
||||||
|
logger.info(f"[AUDIT] LLM egress completed for session {self.session_id}")
|
||||||
|
|
||||||
|
def on_llm_error(self, error: Exception, **kwargs) -> None:
|
||||||
|
logger.error(f"[AUDIT] LLM error for session {self.session_id}: {str(error)}")
|
||||||
|
|
||||||
|
class VertexAILangchainAdapter:
|
||||||
|
def __init__(self):
|
||||||
|
self.llm = VertexAI(
|
||||||
|
model_name=config.VERTEX_AI_MODEL,
|
||||||
|
project_id=config.VERTEX_AI_PROJECT,
|
||||||
|
location=config.VERTEX_AI_LOCATION,
|
||||||
|
max_output_tokens=256,
|
||||||
|
temperature=0.2,
|
||||||
|
top_p=0.8,
|
||||||
|
top_k=40,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def generate(self, prompt: str, session_id: str, metadata: Optional[dict] = None) -> str:
|
||||||
|
import asyncio
|
||||||
|
loop = asyncio.get_event_loop()
|
||||||
|
|
||||||
|
callback_handler = AuditCallbackHandler(session_id, metadata)
|
||||||
|
|
||||||
|
def _sync_generate():
|
||||||
|
result = self.llm.generate(
|
||||||
|
prompts=[prompt],
|
||||||
|
callbacks=[callback_handler]
|
||||||
|
)
|
||||||
|
return result.generations[0][0].text
|
||||||
|
|
||||||
|
return await loop.run_in_executor(None, _sync_generate)
|
||||||
|
|
||||||
|
async def stream_generate(self, prompt: str, session_id: str, metadata: Optional[dict] = None) -> AsyncGenerator[str, None]:
|
||||||
|
import asyncio
|
||||||
|
loop = asyncio.get_event_loop()
|
||||||
|
callback_handler = AuditCallbackHandler(session_id, metadata)
|
||||||
|
|
||||||
|
def _sync_stream():
|
||||||
|
return self.llm.stream(prompt, callbacks=[callback_handler])
|
||||||
|
|
||||||
|
stream = await loop.run_in_executor(None, _sync_stream)
|
||||||
|
for chunk in stream:
|
||||||
|
yield chunk
|
||||||
|
|
||||||
|
def get_llm_adapter() -> VertexAILangchainAdapter:
|
||||||
|
return VertexAILangchainAdapter()
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
import redis
|
||||||
|
import logging
|
||||||
|
from backend.implementation import config
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class RedisClient:
|
||||||
|
"""
|
||||||
|
Singleton Redis client for managing session state and consult_mode.
|
||||||
|
"""
|
||||||
|
_instance = None
|
||||||
|
|
||||||
|
def __new__(cls):
|
||||||
|
if cls._instance is None:
|
||||||
|
cls._instance = super(RedisClient, cls).__new__(cls)
|
||||||
|
try:
|
||||||
|
cls._instance.client = redis.Redis(
|
||||||
|
host=config.REDIS_HOST,
|
||||||
|
port=config.REDIS_PORT,
|
||||||
|
db=config.REDIS_DB,
|
||||||
|
decode_responses=True
|
||||||
|
)
|
||||||
|
logger.info("Connected to Redis at %s:%s", config.REDIS_HOST, config.REDIS_PORT)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("Failed to connect to Redis: %s", e)
|
||||||
|
cls._instance.client = None
|
||||||
|
return cls._instance
|
||||||
|
|
||||||
|
def get(self, key: str):
|
||||||
|
return self.client.get(key) if self.client else None
|
||||||
|
|
||||||
|
def set(self, key: str, value: str, ex: int = None):
|
||||||
|
if self.client:
|
||||||
|
self.client.set(key, value, ex=ex)
|
||||||
|
|
||||||
|
def exists(self, key: str) -> bool:
|
||||||
|
return bool(self.client.exists(key)) if self.client else False
|
||||||
|
|
||||||
|
def get_redis_client() -> RedisClient:
|
||||||
|
return RedisClient()
|
||||||
@@ -0,0 +1,121 @@
|
|||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
from typing import Any
|
||||||
|
import numpy as np
|
||||||
|
import requests
|
||||||
|
|
||||||
|
|
||||||
|
class TritonAdapter:
|
||||||
|
def __init__(self, endpoint_url: str, timeout: float = 60.0):
|
||||||
|
self.endpoint_url = endpoint_url.rstrip("/")
|
||||||
|
self.timeout = timeout
|
||||||
|
|
||||||
|
|
||||||
|
async def close(self):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
async def infer(
|
||||||
|
self, model_name: str, inputs: dict, outputs: list[str] | None = None
|
||||||
|
) -> dict:
|
||||||
|
return await asyncio.to_thread(
|
||||||
|
self._infer_sync, model_name, inputs, outputs
|
||||||
|
)
|
||||||
|
|
||||||
|
def _infer_sync(
|
||||||
|
self, model_name: str, inputs: dict, outputs: list[str] | None = None
|
||||||
|
) -> dict:
|
||||||
|
metadata_inputs = []
|
||||||
|
binary_chunks = []
|
||||||
|
|
||||||
|
for name, spec in inputs.items():
|
||||||
|
data = spec["data"]
|
||||||
|
shape = spec.get("shape", [])
|
||||||
|
datatype = spec.get("datatype", "FP32")
|
||||||
|
|
||||||
|
arr = np.asarray(data, dtype=np.float32)
|
||||||
|
binary = arr.tobytes()
|
||||||
|
|
||||||
|
metadata_inputs.append(
|
||||||
|
{
|
||||||
|
"name": name,
|
||||||
|
"shape": shape,
|
||||||
|
"datatype": datatype,
|
||||||
|
"parameters": {"binary_data_size": len(binary)},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
binary_chunks.append(binary)
|
||||||
|
|
||||||
|
metadata_outputs = [{"name": o} for o in (outputs or [])]
|
||||||
|
metadata = {
|
||||||
|
"inputs": metadata_inputs,
|
||||||
|
"outputs": metadata_outputs,
|
||||||
|
}
|
||||||
|
|
||||||
|
metadata_bytes = json.dumps(metadata).encode("utf-8")
|
||||||
|
body = metadata_bytes + b"".join(binary_chunks)
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
"Inference-Header-Content-Length": str(len(metadata_bytes)),
|
||||||
|
"Content-Type": "application/octet-stream",
|
||||||
|
}
|
||||||
|
|
||||||
|
url = f"{self.endpoint_url}/v2/models/{model_name}/infer"
|
||||||
|
response = requests.post(url, data=body, headers=headers, timeout=self.timeout)
|
||||||
|
response.raise_for_status()
|
||||||
|
return self._parse_binary_response(response.headers, response.content)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _parse_binary_response(headers: dict, body: bytes) -> dict:
|
||||||
|
header_len = int(headers.get("Inference-Header-Content-Length", "0"))
|
||||||
|
metadata = json.loads(body[:header_len].decode("utf-8"))
|
||||||
|
|
||||||
|
result = {}
|
||||||
|
offset = 0
|
||||||
|
for output in metadata.get("outputs", []):
|
||||||
|
name = output["name"]
|
||||||
|
shape = output.get("shape", [])
|
||||||
|
params = output.get("parameters", {})
|
||||||
|
binary_size = params.get("binary_data_size", 0)
|
||||||
|
|
||||||
|
if binary_size > 0:
|
||||||
|
chunk = body[header_len + offset : header_len + offset + binary_size]
|
||||||
|
arr = np.frombuffer(chunk, dtype=np.float32).reshape(shape)
|
||||||
|
result[name] = arr.tolist()
|
||||||
|
offset += binary_size
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
async def model_ready(self, model_name: str) -> bool:
|
||||||
|
return await asyncio.to_thread(self._model_ready_sync, model_name)
|
||||||
|
|
||||||
|
def _model_ready_sync(self, model_name: str) -> bool:
|
||||||
|
url = f"{self.endpoint_url}/v2/models/{model_name}"
|
||||||
|
response = requests.get(url, timeout=self.timeout)
|
||||||
|
if response.status_code == 404:
|
||||||
|
return False
|
||||||
|
response.raise_for_status()
|
||||||
|
data = response.json()
|
||||||
|
return data.get("ready", False)
|
||||||
|
|
||||||
|
async def list_models(self) -> list[dict]:
|
||||||
|
return await asyncio.to_thread(self._list_models_sync)
|
||||||
|
|
||||||
|
# def _list_models_sync(self) -> list[dict]:
|
||||||
|
# url = f"{self.endpoint_url}/v2/models"
|
||||||
|
# response = requests.get(url, timeout=self.timeout)
|
||||||
|
# response.raise_for_status()
|
||||||
|
# data = response.json()
|
||||||
|
# return data.get("models", [])
|
||||||
|
|
||||||
|
def _list_models_sync(self) -> list[dict]:
|
||||||
|
# 1. Change the endpoint to Triton's repository index path
|
||||||
|
url = f"{self.endpoint_url}/v2/repository/index"
|
||||||
|
|
||||||
|
# 2. Change requests.get to requests.post with an empty json payload {}
|
||||||
|
response = requests.post(url, json={}, timeout=self.timeout)
|
||||||
|
response.raise_for_status()
|
||||||
|
|
||||||
|
data = response.json()
|
||||||
|
# KServe v2 returns a list directly: [{"name": "model_a", "version": "1", "state": "READY"}]
|
||||||
|
return data
|
||||||
@@ -0,0 +1,323 @@
|
|||||||
|
import asyncio
|
||||||
|
import io
|
||||||
|
import base64
|
||||||
|
import uuid
|
||||||
|
import logging
|
||||||
|
import numpy as np
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from data.spec.schemas import (
|
||||||
|
AnalysisJobSubmit, JobStatus, JobResult, PipelineStep,
|
||||||
|
StepEvent, ModelCatalog, ModelRegistrationResult,
|
||||||
|
HealthStatus,
|
||||||
|
)
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
from backend.implementation.preprocessing.clahe import apply_clahe
|
||||||
|
from backend.implementation.preprocessing.tensor_prep import (
|
||||||
|
prepare_angle_tensor,
|
||||||
|
prepare_inflammation_tensor,
|
||||||
|
prepare_segmentation_tensor,
|
||||||
|
)
|
||||||
|
from backend.implementation.postprocessing.measurement import calculate_thickness
|
||||||
|
from backend.implementation.postprocessing.severity import calculate_severity
|
||||||
|
from backend.implementation.postprocessing.overlay import create_overlay
|
||||||
|
from backend.implementation.postprocessing.calibration import (
|
||||||
|
calibration_config_from_params,
|
||||||
|
interpret_angle_logits,
|
||||||
|
interpret_inflammation_logits,
|
||||||
|
)
|
||||||
|
from backend.implementation.config import (
|
||||||
|
get_model_name,
|
||||||
|
get_segmentation_model,
|
||||||
|
get_angle_type,
|
||||||
|
TRITON_ENDPOINT,
|
||||||
|
)
|
||||||
|
from backend.implementation.adapters.triton_adapter import TritonAdapter
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_job_registry: dict[str, dict] = {}
|
||||||
|
_job_lock = asyncio.Lock()
|
||||||
|
|
||||||
|
|
||||||
|
def _interpret_angle_result(result: dict, params: dict | None = None) -> dict:
|
||||||
|
logits = result.get("logits", [])
|
||||||
|
if not logits:
|
||||||
|
raise ValueError("Empty angle logits")
|
||||||
|
config = calibration_config_from_params(params)
|
||||||
|
return interpret_angle_logits(logits, config)
|
||||||
|
|
||||||
|
|
||||||
|
def _interpret_inflammation_result(result: dict, params: dict | None = None) -> dict:
|
||||||
|
logits = result.get("logits", [])
|
||||||
|
if not logits:
|
||||||
|
raise ValueError("Empty inflammation logits")
|
||||||
|
config = calibration_config_from_params(params)
|
||||||
|
return interpret_inflammation_logits(logits, config)
|
||||||
|
|
||||||
|
|
||||||
|
def _process_segmentation_result(result: dict, angle_class: str) -> tuple:
|
||||||
|
logits = result.get("logits", [])
|
||||||
|
if not logits:
|
||||||
|
raise ValueError("Empty segmentation logits")
|
||||||
|
logits_arr = np.array(logits)
|
||||||
|
if logits_arr.ndim < 3:
|
||||||
|
raise ValueError("Unexpected segmentation output shape")
|
||||||
|
preds = logits_arr.argmax(axis=1)[0]
|
||||||
|
angle_type = get_angle_type(angle_class)
|
||||||
|
if angle_type == "sup":
|
||||||
|
class_map = {
|
||||||
|
0: "background", 1: "effusion", 2: "fat", 3: "fat-pat",
|
||||||
|
4: "femur", 5: "synovium", 6: "tendon",
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
class_map = {
|
||||||
|
0: "background", 1: "fat", 2: "tendon", 3: "muscle",
|
||||||
|
4: "femur", 5: "artery", 6: "baker's cyst",
|
||||||
|
}
|
||||||
|
masks = {}
|
||||||
|
for class_id, class_name in class_map.items():
|
||||||
|
masks[class_name] = (preds == class_id).astype(np.uint8)
|
||||||
|
return preds, masks
|
||||||
|
|
||||||
|
|
||||||
|
async def _get_triton_adapter() -> TritonAdapter:
|
||||||
|
return TritonAdapter(endpoint_url=TRITON_ENDPOINT)
|
||||||
|
|
||||||
|
|
||||||
|
def _encode_image_to_base64(image_pil: Image.Image) -> str:
|
||||||
|
buffered = io.BytesIO()
|
||||||
|
image_pil.save(buffered, format="PNG")
|
||||||
|
return base64.b64encode(buffered.getvalue()).decode()
|
||||||
|
|
||||||
|
|
||||||
|
async def _run_pipeline(image_pil: Image.Image, session_id: str, params: dict, model_versions: dict | None = None) -> dict:
|
||||||
|
enhanced_pil = apply_clahe(image_pil)
|
||||||
|
angle_tensor = prepare_angle_tensor(image_pil)
|
||||||
|
inflammation_tensor = prepare_inflammation_tensor(image_pil)
|
||||||
|
segmentation_tensor = prepare_segmentation_tensor(image_pil)
|
||||||
|
|
||||||
|
triton = await _get_triton_adapter()
|
||||||
|
angle_model = get_model_name("angle", model_versions)
|
||||||
|
angle_result = await triton.infer(
|
||||||
|
model_name=angle_model,
|
||||||
|
inputs={"input": {"data": angle_tensor.tolist(), "shape": list(angle_tensor.shape), "datatype": "FP32"}},
|
||||||
|
)
|
||||||
|
angle_interpreted = _interpret_angle_result(angle_result, params)
|
||||||
|
|
||||||
|
result = {
|
||||||
|
"angle": {
|
||||||
|
"class": angle_interpreted["class"],
|
||||||
|
"confidence": angle_interpreted["confidence"],
|
||||||
|
"calibration": angle_interpreted["calibration"],
|
||||||
|
},
|
||||||
|
"models_used": {"angle": angle_model},
|
||||||
|
}
|
||||||
|
|
||||||
|
if angle_interpreted["class"] in ("post-trans", "sup-up-long"):
|
||||||
|
inflam_model = get_model_name("inflammation", model_versions)
|
||||||
|
inflammation_result = await triton.infer(
|
||||||
|
model_name=inflam_model,
|
||||||
|
inputs={"input": {"data": inflammation_tensor.tolist(), "shape": list(inflammation_tensor.shape), "datatype": "FP32"}},
|
||||||
|
)
|
||||||
|
inflammation_interpreted = _interpret_inflammation_result(inflammation_result, params)
|
||||||
|
result["inflammation"] = {
|
||||||
|
"detected": inflammation_interpreted["detected"],
|
||||||
|
"confidence": inflammation_interpreted["confidence"],
|
||||||
|
"calibration": inflammation_interpreted["calibration"],
|
||||||
|
}
|
||||||
|
result["models_used"]["inflammation"] = inflam_model
|
||||||
|
|
||||||
|
if inflammation_interpreted["detected"]:
|
||||||
|
seg_model_name = get_segmentation_model(angle_interpreted["class"], model_versions)
|
||||||
|
seg_result = await triton.infer(
|
||||||
|
model_name=seg_model_name,
|
||||||
|
inputs={"input": {"data": segmentation_tensor.tolist(), "shape": list(segmentation_tensor.shape), "datatype": "FP32"}},
|
||||||
|
)
|
||||||
|
preds, masks = _process_segmentation_result(seg_result, angle_interpreted["class"])
|
||||||
|
angle_type = get_angle_type(angle_interpreted["class"])
|
||||||
|
measurement = calculate_thickness(masks, image_pil.size)
|
||||||
|
severity = calculate_severity(masks, image_pil.size)
|
||||||
|
segmented_overlay = create_overlay(image_pil, masks, measurement, angle_type)
|
||||||
|
|
||||||
|
result.update({
|
||||||
|
"measurement": measurement,
|
||||||
|
"severity": severity,
|
||||||
|
"segmentation": {
|
||||||
|
"performed": True,
|
||||||
|
"classes_detected": [k for k, v in masks.items() if np.sum(v) > 0],
|
||||||
|
"angle_type": angle_type,
|
||||||
|
},
|
||||||
|
"images": {
|
||||||
|
"enhanced": _encode_image_to_base64(enhanced_pil),
|
||||||
|
"segmented": _encode_image_to_base64(segmented_overlay),
|
||||||
|
},
|
||||||
|
})
|
||||||
|
result["models_used"]["segmentation"] = seg_model_name
|
||||||
|
else:
|
||||||
|
from backend.implementation.pipeline.cv_spec_pipeline import (
|
||||||
|
build_segmentation_skipped,
|
||||||
|
build_severity_zero,
|
||||||
|
)
|
||||||
|
|
||||||
|
result["segmentation"] = build_segmentation_skipped("no_inflammation")
|
||||||
|
result["severity"] = build_severity_zero("no_inflammation")
|
||||||
|
result["images"] = {"enhanced": _encode_image_to_base64(enhanced_pil)}
|
||||||
|
else:
|
||||||
|
from backend.implementation.pipeline.cv_spec_pipeline import (
|
||||||
|
build_segmentation_skipped,
|
||||||
|
build_severity_zero,
|
||||||
|
)
|
||||||
|
|
||||||
|
result["segmentation"] = build_segmentation_skipped("angle_only")
|
||||||
|
result["severity"] = build_severity_zero("angle_only")
|
||||||
|
result["images"] = {"enhanced": _encode_image_to_base64(enhanced_pil)}
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
async def submit_sync(session_id: str, params: dict, model_versions: dict | None = None) -> JobResult:
|
||||||
|
if "local_image_path" in params:
|
||||||
|
image_pil = Image.open(params["local_image_path"]).convert("RGB")
|
||||||
|
elif "local_image_bytes" in params:
|
||||||
|
image_pil = Image.open(io.BytesIO(params["local_image_bytes"])).convert("RGB")
|
||||||
|
else:
|
||||||
|
from backend.implementation.session import service as session_service
|
||||||
|
frame_metadata = await session_service.get_frame(session_id, params.get("frame_id"))
|
||||||
|
raise NotImplementedError("S3 frame retrieval not yet integrated; use local_image_path for testing")
|
||||||
|
|
||||||
|
pipeline_result = await _run_pipeline(image_pil, session_id, params, model_versions)
|
||||||
|
job_id = str(uuid.uuid4())
|
||||||
|
return JobResult(
|
||||||
|
job_id=job_id,
|
||||||
|
session_id=session_id,
|
||||||
|
status="completed",
|
||||||
|
result=pipeline_result,
|
||||||
|
duration_ms=0,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def submit_job(session_id: str, params: dict, model_versions: dict | None = None) -> str:
|
||||||
|
job_id = str(uuid.uuid4())
|
||||||
|
async with _job_lock:
|
||||||
|
_job_registry[job_id] = {
|
||||||
|
"session_id": session_id,
|
||||||
|
"params": params,
|
||||||
|
"model_versions": model_versions,
|
||||||
|
"status": "queued",
|
||||||
|
"result": None,
|
||||||
|
"steps": [],
|
||||||
|
"created_at": datetime.now(),
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _background():
|
||||||
|
try:
|
||||||
|
await push_step_event(job_id, {
|
||||||
|
"step_id": str(uuid.uuid4()),
|
||||||
|
"job_id": job_id,
|
||||||
|
"event_type": "progress",
|
||||||
|
"task_type": "analysis",
|
||||||
|
"status": "running",
|
||||||
|
})
|
||||||
|
async with _job_lock:
|
||||||
|
_job_registry[job_id]["status"] = "running"
|
||||||
|
result = await submit_sync(session_id, params, model_versions)
|
||||||
|
async with _job_lock:
|
||||||
|
_job_registry[job_id].update({
|
||||||
|
"status": "completed",
|
||||||
|
"result": result.model_dump(),
|
||||||
|
})
|
||||||
|
await push_step_event(job_id, {
|
||||||
|
"step_id": str(uuid.uuid4()),
|
||||||
|
"job_id": job_id,
|
||||||
|
"event_type": "completed",
|
||||||
|
"task_type": "analysis",
|
||||||
|
"status": "completed",
|
||||||
|
"data": {"job_result": result.model_dump()},
|
||||||
|
})
|
||||||
|
except Exception as exc:
|
||||||
|
logger.exception(f"Job {job_id} failed")
|
||||||
|
async with _job_lock:
|
||||||
|
_job_registry[job_id].update({
|
||||||
|
"status": "failed",
|
||||||
|
"result": {"error": str(exc)},
|
||||||
|
})
|
||||||
|
await push_step_event(job_id, {
|
||||||
|
"step_id": str(uuid.uuid4()),
|
||||||
|
"job_id": job_id,
|
||||||
|
"event_type": "failed",
|
||||||
|
"task_type": "analysis",
|
||||||
|
"status": "failed",
|
||||||
|
"data": {"error": str(exc)},
|
||||||
|
})
|
||||||
|
|
||||||
|
asyncio.create_task(_background())
|
||||||
|
return job_id
|
||||||
|
|
||||||
|
|
||||||
|
async def job_status(job_id: str) -> JobStatus:
|
||||||
|
async with _job_lock:
|
||||||
|
job = _job_registry.get(job_id)
|
||||||
|
if not job:
|
||||||
|
raise LookupError(f"Job {job_id} not found")
|
||||||
|
return JobStatus(
|
||||||
|
job_id=job_id,
|
||||||
|
session_id=job["session_id"],
|
||||||
|
status=job["status"],
|
||||||
|
result=job.get("result"),
|
||||||
|
steps=job.get("steps", []),
|
||||||
|
created_at=job["created_at"],
|
||||||
|
updated_at=datetime.now(),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def job_steps(job_id: str) -> list[PipelineStep]:
|
||||||
|
async with _job_lock:
|
||||||
|
job = _job_registry.get(job_id)
|
||||||
|
if not job:
|
||||||
|
raise LookupError(f"Job {job_id} not found")
|
||||||
|
return job.get("steps", [])
|
||||||
|
|
||||||
|
|
||||||
|
async def list_registered_models() -> ModelCatalog:
|
||||||
|
return ModelCatalog(models=[], total=0)
|
||||||
|
|
||||||
|
|
||||||
|
async def register_model(model_id: str, file: Any) -> ModelRegistrationResult:
|
||||||
|
raise NotImplementedError("Model registration not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def health() -> HealthStatus:
|
||||||
|
try:
|
||||||
|
triton = await _get_triton_adapter()
|
||||||
|
models = await triton.list_models()
|
||||||
|
ready = any(m.get("state") == "READY" for m in models)
|
||||||
|
status = "ok" if ready else "degraded"
|
||||||
|
return HealthStatus(
|
||||||
|
status=status,
|
||||||
|
version="0.1.0",
|
||||||
|
dependencies={"triton": str(ready)},
|
||||||
|
uptime_seconds=0.0,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"Health check failed: {exc}")
|
||||||
|
return HealthStatus(status="error", version="0.1.0", dependencies={"triton": "False"}, uptime_seconds=0.0)
|
||||||
|
|
||||||
|
|
||||||
|
async def push_step_event(job_id: str, event: dict) -> None:
|
||||||
|
from backend.api.analysis_api import _event_queues, _queue_lock
|
||||||
|
async with _queue_lock:
|
||||||
|
if job_id not in _event_queues:
|
||||||
|
_event_queues[job_id] = asyncio.Queue()
|
||||||
|
step_event = StepEvent(
|
||||||
|
step_id=event.get("step_id", ""),
|
||||||
|
job_id=job_id,
|
||||||
|
event_type=event.get("event_type", "progress"),
|
||||||
|
task_type=event.get("task_type", ""),
|
||||||
|
status=event.get("status", "running"),
|
||||||
|
data=event.get("data"),
|
||||||
|
timestamp=datetime.now(),
|
||||||
|
)
|
||||||
|
await _event_queues[job_id].put(step_event)
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
from data.spec.schemas import LoginRequest, Token, UserProfile, UserUpdateRequest
|
||||||
|
|
||||||
|
|
||||||
|
async def login(username: str, password: str) -> Token:
|
||||||
|
raise NotImplementedError("Auth service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def logout(token: str) -> None:
|
||||||
|
raise NotImplementedError("Auth service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def refresh(refresh_token: str) -> Token:
|
||||||
|
raise NotImplementedError("Auth service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def me(token: str) -> UserProfile:
|
||||||
|
raise NotImplementedError("Auth service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def update_me(token: str, updates: dict) -> UserProfile:
|
||||||
|
raise NotImplementedError("Auth service not yet implemented")
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Dict
|
||||||
|
|
||||||
|
SECRETS_DIR = Path(__file__).resolve().parent.parent.parent.parent.parent.parent / "secrets"
|
||||||
|
|
||||||
|
def _load_secret(name: str, filename: str) -> str:
|
||||||
|
file_path = SECRETS_DIR / filename
|
||||||
|
env_file = os.getenv(f"{name}_FILE")
|
||||||
|
if env_file:
|
||||||
|
resolved = Path(env_file)
|
||||||
|
if resolved.exists():
|
||||||
|
with open(resolved, "r", encoding="utf-8") as f:
|
||||||
|
return f.read().strip()
|
||||||
|
if file_path.exists():
|
||||||
|
with open(file_path, "r", encoding="utf-8") as f:
|
||||||
|
return f.read().strip()
|
||||||
|
raise RuntimeError(
|
||||||
|
f"Required secret {name} not found at {file_path} or via {name}_FILE env var"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Endpoints (environment-provided, no hardcoded fallback for production)
|
||||||
|
MODAL_MEDGEMMA_ENDPOINT = os.getenv("MODAL_MEDGEMMA_ENDPOINT")
|
||||||
|
VERTEX_AI_GEMINI_ENDPOINT = os.getenv("VERTEX_AI_GEMINI_ENDPOINT")
|
||||||
|
|
||||||
|
# Secrets (must be present in PILOT_PROJECT/secrets or env)
|
||||||
|
GCP_ACCESS_TOKEN = _load_secret("GCP_ACCESS_TOKEN", "gcp_access_token.txt")
|
||||||
|
MEDGEMMA_API_KEY = _load_secret("MEDGEMMA_API_KEY", "modal_api_key.txt")
|
||||||
|
|
||||||
|
PROJECT_ID = os.getenv("VERTEX_AI_PROJECT", "vkist-project")
|
||||||
|
LOCATION = os.getenv("VERTEX_AI_LOCATION", "asia-southeast1")
|
||||||
|
|
||||||
|
TRITON_ENDPOINT = os.getenv("TRITON_ENDPOINT", "http://localhost:8000")
|
||||||
|
TEMP_DIR = os.getenv("TEMP_DIR", "/tmp/analysis_jobs")
|
||||||
|
|
||||||
|
# LLM Configuration
|
||||||
|
VERTEX_AI_PROJECT = os.getenv("VERTEX_AI_PROJECT", "vkist-project")
|
||||||
|
VERTEX_AI_LOCATION = os.getenv("VERTEX_AI_LOCATION", "asia-southeast1")
|
||||||
|
VERTEX_AI_MODEL = os.getenv("VERTEX_AI_MODEL", "medgemma")
|
||||||
|
|
||||||
|
# Redis Configuration
|
||||||
|
REDIS_HOST = os.getenv("REDIS_HOST", "localhost")
|
||||||
|
REDIS_PORT = int(os.getenv("REDIS_PORT", "6379"))
|
||||||
|
REDIS_DB = int(os.getenv("REDIS_DB", "0"))
|
||||||
|
|
||||||
|
DEFAULT_MODEL_VERSIONS = {
|
||||||
|
"angle": "angle_classify_convnext_tiny",
|
||||||
|
"inflammation": "inflammation_model_efficientnet_b0_ultrasound_2_cls",
|
||||||
|
"segmentation_sup": "segmentation_model_unet_resnet101",
|
||||||
|
"segmentation_post": "segmentation_model_post_deeplabv3_resnet101",
|
||||||
|
}
|
||||||
|
|
||||||
|
CLAHE_CLIP_LIMIT = float(os.getenv("CLAHE_CLIP_LIMIT", "2.0"))
|
||||||
|
CLAHE_TILE_SIZE = tuple(int(x) for x in os.getenv("CLAHE_TILE_SIZE", "8,8").split(","))
|
||||||
|
|
||||||
|
|
||||||
|
def get_model_name(task: str, model_versions: Dict[str, str] | None = None) -> str:
|
||||||
|
if model_versions and task in model_versions:
|
||||||
|
return model_versions[task]
|
||||||
|
return DEFAULT_MODEL_VERSIONS.get(task, task)
|
||||||
|
|
||||||
|
|
||||||
|
def get_angle_type(angle_class: str) -> str:
|
||||||
|
if angle_class in ("sup-trans-flex", "sup-up-long"):
|
||||||
|
return "sup"
|
||||||
|
if angle_class == "post-trans":
|
||||||
|
return "post"
|
||||||
|
return "other"
|
||||||
|
|
||||||
|
|
||||||
|
def get_segmentation_model(angle_class: str, model_versions: Dict[str, str] | None = None) -> str:
|
||||||
|
angle_type = get_angle_type(angle_class)
|
||||||
|
task = "segmentation_sup" if angle_type == "sup" else "segmentation_post"
|
||||||
|
return get_model_name(task, model_versions)
|
||||||
|
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
from data.spec.schemas import IngestionRecord, RecordDetail
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
async def list_records(user_id: str) -> list[IngestionRecord]:
|
||||||
|
raise NotImplementedError("Ingestion history service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def get_record(record_id: str) -> RecordDetail:
|
||||||
|
raise NotImplementedError("Ingestion history service not yet implemented")
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
from data.spec.schemas import NotificationItem, NotificationPreferences
|
||||||
|
|
||||||
|
|
||||||
|
async def list_notifications(user_id: str, filters: dict | None = None) -> list[NotificationItem]:
|
||||||
|
raise NotImplementedError("Notification service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def mark_read(notification_id: str) -> None:
|
||||||
|
raise NotImplementedError("Notification service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def set_preferences(user_id: str, prefs: dict) -> None:
|
||||||
|
raise NotImplementedError("Notification service not yet implemented")
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
from data.spec.schemas import Patient, PatientCreate, PatientListResponse
|
||||||
|
|
||||||
|
|
||||||
|
async def list_patients(user_id: str) -> list[Patient]:
|
||||||
|
raise NotImplementedError("Patient service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def create_patient(data: dict) -> Patient:
|
||||||
|
raise NotImplementedError("Patient service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def get_patient(patient_id: str) -> Patient:
|
||||||
|
raise NotImplementedError("Patient service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def list_sessions(patient_id: str) -> list[dict]:
|
||||||
|
raise NotImplementedError("Patient service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def ingestion_history(patient_id: str) -> list[dict]:
|
||||||
|
raise NotImplementedError("Patient service not yet implemented")
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
"""CV inference orchestration (Sprint 1–2 spec)."""
|
||||||
|
|
||||||
|
from backend.implementation.pipeline.cv_spec_pipeline import (
|
||||||
|
BRANCH_ANGLE_CLASSES,
|
||||||
|
build_segmentation_skipped,
|
||||||
|
build_severity_zero,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"BRANCH_ANGLE_CLASSES",
|
||||||
|
"build_segmentation_skipped",
|
||||||
|
"build_severity_zero",
|
||||||
|
]
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
"""Shared CV pipeline helpers — Sprint 1–2 architecture spec §7."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
# Angles that may run inflammation → conditional segmentation.
|
||||||
|
BRANCH_ANGLE_CLASSES = frozenset({"post-trans", "sup-up-long"})
|
||||||
|
|
||||||
|
|
||||||
|
def build_severity_zero(reason: str) -> dict:
|
||||||
|
descriptions = {
|
||||||
|
"angle_only": "Góc quét không yêu cầu phân đoạn viêm",
|
||||||
|
"no_inflammation": "Không phát hiện viêm — bỏ qua phân đoạn",
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"level": 0,
|
||||||
|
"severity": "Rất nhẹ",
|
||||||
|
"color": "#28a745",
|
||||||
|
"description": descriptions.get(reason, "Không phân đoạn"),
|
||||||
|
"effusion": {"pixels": 0, "ratio": 0.0, "thickness": 0},
|
||||||
|
"synovium": {"pixels": 0, "ratio": 0.0},
|
||||||
|
"combined_score": 0.0,
|
||||||
|
"reason": reason,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_segmentation_skipped(reason: str) -> dict:
|
||||||
|
notes = {
|
||||||
|
"angle_only": "Chỉ phân loại góc — med-lat / sup-trans-flex",
|
||||||
|
"no_inflammation": "Không phát hiện viêm — bỏ qua phân đoạn",
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"performed": False,
|
||||||
|
"reason": reason,
|
||||||
|
"note": notes.get(reason, reason),
|
||||||
|
}
|
||||||
@@ -0,0 +1,230 @@
|
|||||||
|
"""Temperature-scaled softmax, entropy guardrails, and risk-first prediction payloads."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
ANGLE_CLASSES = ["med-lat", "post-trans", "sup-trans-flex", "sup-up-long"]
|
||||||
|
INFLAMMATION_CLASSES = ["no_inflammation", "inflammation"]
|
||||||
|
CALIBRATION_TIERS = frozenset({"aggressive", "standard", "conservative"})
|
||||||
|
# Legacy API aliases
|
||||||
|
CALIBRATION_MODES = CALIBRATION_TIERS | frozenset({"screening", "diagnostic"})
|
||||||
|
|
||||||
|
TIER_RECOMMENDED_T = {
|
||||||
|
"aggressive": 0.7,
|
||||||
|
"standard": 1.4,
|
||||||
|
"conservative": 2.2,
|
||||||
|
# legacy → tier
|
||||||
|
"screening": 2.2,
|
||||||
|
"diagnostic": 1.4,
|
||||||
|
}
|
||||||
|
|
||||||
|
TIER_BOUNDARY_AGGRESSIVE_MAX = (0.7 + 1.4) / 2
|
||||||
|
TIER_BOUNDARY_STANDARD_MAX = (1.4 + 2.2) / 2
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class CalibrationConfig:
|
||||||
|
"""User-adjustable calibration context (maps to UI mode / clinical prior)."""
|
||||||
|
|
||||||
|
temperature: float = 1.4
|
||||||
|
mode: str = "standard"
|
||||||
|
clinical_suspicion: float = 0.0
|
||||||
|
alpha_margin: float = 0.05
|
||||||
|
ood_entropy_threshold: float = 0.88
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
self.clinical_suspicion = float(np.clip(self.clinical_suspicion, 0.0, 1.0))
|
||||||
|
if self.mode not in CALIBRATION_TIERS:
|
||||||
|
if self.mode in ("screening",):
|
||||||
|
self.mode = "conservative"
|
||||||
|
elif self.mode in ("diagnostic",):
|
||||||
|
self.mode = "standard"
|
||||||
|
else:
|
||||||
|
self.mode = "standard"
|
||||||
|
if self.temperature <= 0:
|
||||||
|
self.temperature = 1.0
|
||||||
|
|
||||||
|
|
||||||
|
def logits_to_array(logits: Any) -> np.ndarray:
|
||||||
|
arr = np.asarray(logits, dtype=np.float32).reshape(-1)
|
||||||
|
if arr.size == 0:
|
||||||
|
raise ValueError("Empty logits")
|
||||||
|
return arr
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_tier_from_temperature(temperature: float) -> str:
|
||||||
|
if temperature <= TIER_BOUNDARY_AGGRESSIVE_MAX:
|
||||||
|
return "aggressive"
|
||||||
|
if temperature <= TIER_BOUNDARY_STANDARD_MAX:
|
||||||
|
return "standard"
|
||||||
|
return "conservative"
|
||||||
|
|
||||||
|
|
||||||
|
def effective_temperature(config: CalibrationConfig) -> float:
|
||||||
|
if config.temperature > 0:
|
||||||
|
return max(0.25, float(config.temperature))
|
||||||
|
return TIER_RECOMMENDED_T.get(config.mode, 1.4)
|
||||||
|
|
||||||
|
|
||||||
|
def temperature_scaled_softmax(logits: np.ndarray, temperature: float) -> np.ndarray:
|
||||||
|
scaled = logits / max(temperature, 1e-6)
|
||||||
|
shifted = scaled - np.max(scaled)
|
||||||
|
exp = np.exp(shifted)
|
||||||
|
return exp / np.sum(exp)
|
||||||
|
|
||||||
|
|
||||||
|
def shannon_entropy(probs: np.ndarray) -> float:
|
||||||
|
safe = np.clip(probs, 1e-12, 1.0)
|
||||||
|
return float(-np.sum(safe * np.log(safe)))
|
||||||
|
|
||||||
|
|
||||||
|
def normalized_entropy(probs: np.ndarray) -> float:
|
||||||
|
if probs.size <= 1:
|
||||||
|
return 0.0
|
||||||
|
return shannon_entropy(probs) / float(np.log(probs.size))
|
||||||
|
|
||||||
|
|
||||||
|
def ambiguous_class_set(probs: np.ndarray, class_names: list[str], alpha_margin: float) -> list[str]:
|
||||||
|
idx_sorted = np.argsort(probs)[::-1]
|
||||||
|
max_prob = float(probs[idx_sorted[0]])
|
||||||
|
return [class_names[i] for i in idx_sorted if float(probs[i]) >= max_prob - alpha_margin]
|
||||||
|
|
||||||
|
|
||||||
|
def estimate_misclassification_rate(max_prob: float) -> float:
|
||||||
|
"""Placeholder empirical mapping until validation-set calibration bins are wired."""
|
||||||
|
if max_prob >= 0.95:
|
||||||
|
return 0.05
|
||||||
|
if max_prob >= 0.90:
|
||||||
|
return 0.08
|
||||||
|
if max_prob >= 0.85:
|
||||||
|
return 0.12
|
||||||
|
if max_prob >= 0.75:
|
||||||
|
return 0.18
|
||||||
|
if max_prob >= 0.65:
|
||||||
|
return 0.25
|
||||||
|
return 0.35
|
||||||
|
|
||||||
|
|
||||||
|
def _risk_framing_vi(
|
||||||
|
predicted_class: str,
|
||||||
|
class_names: list[str],
|
||||||
|
probs: np.ndarray,
|
||||||
|
decision_state: str,
|
||||||
|
error_rate: float,
|
||||||
|
ambiguous_set: list[str],
|
||||||
|
norm_entropy: float,
|
||||||
|
) -> str:
|
||||||
|
if decision_state == "ood_warning":
|
||||||
|
return (
|
||||||
|
"Mô hình chưa được huấn luyện với loại ảnh tương tự, nên kết quả AI có thể không đáng tin. "
|
||||||
|
"Hãy kiểm tra chất lượng ảnh và đối chiếu lâm sàng trước khi dựa vào nhãn tự động."
|
||||||
|
)
|
||||||
|
if decision_state == "ambiguous":
|
||||||
|
alt = ", ".join(c for c in ambiguous_set if c != predicted_class)
|
||||||
|
return (
|
||||||
|
f"Dự đoán chính: {predicted_class}. Tập mơ hồ (α): {', '.join(ambiguous_set)}"
|
||||||
|
+ (f" — các lựa chọn khả dĩ gồm {alt}." if alt else ".")
|
||||||
|
+ " Cần đối chiếu lâm sàng trước khi khóa kết quả."
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
f"Dự đoán: {predicted_class}. "
|
||||||
|
f"Trong các ca có phân bố thống kê tương tự, tỷ lệ phân loại sai ước tính ~{error_rate * 100:.0f}% "
|
||||||
|
f"(entropy chuẩn hóa {norm_entropy:.2f})."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def decision_state_from(probs: np.ndarray, norm_entropy: float, config: CalibrationConfig) -> str:
|
||||||
|
if norm_entropy >= config.ood_entropy_threshold:
|
||||||
|
return "ood_warning"
|
||||||
|
ambiguous = ambiguous_class_set(probs, [str(i) for i in range(probs.size)], config.alpha_margin)
|
||||||
|
if len(ambiguous) > 1:
|
||||||
|
return "ambiguous"
|
||||||
|
return "confident"
|
||||||
|
|
||||||
|
|
||||||
|
def interpret_classification_logits(
|
||||||
|
logits: Any,
|
||||||
|
class_names: list[str],
|
||||||
|
config: CalibrationConfig | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
if len(class_names) == 0:
|
||||||
|
raise ValueError("class_names must not be empty")
|
||||||
|
|
||||||
|
cfg = config or CalibrationConfig()
|
||||||
|
arr = logits_to_array(logits)
|
||||||
|
if arr.size != len(class_names):
|
||||||
|
raise ValueError(f"Expected {len(class_names)} logits, got {arr.size}")
|
||||||
|
|
||||||
|
temperature = effective_temperature(cfg)
|
||||||
|
tier = resolve_tier_from_temperature(temperature)
|
||||||
|
probs = temperature_scaled_softmax(arr, temperature)
|
||||||
|
pred_idx = int(np.argmax(probs))
|
||||||
|
predicted_class = class_names[pred_idx]
|
||||||
|
max_prob = float(probs[pred_idx])
|
||||||
|
entropy = shannon_entropy(probs)
|
||||||
|
norm_entropy = normalized_entropy(probs)
|
||||||
|
ambiguous = ambiguous_class_set(probs, class_names, cfg.alpha_margin)
|
||||||
|
state = decision_state_from(probs, norm_entropy, cfg)
|
||||||
|
error_rate = estimate_misclassification_rate(max_prob)
|
||||||
|
|
||||||
|
risk_vi = _risk_framing_vi(
|
||||||
|
predicted_class,
|
||||||
|
class_names,
|
||||||
|
probs,
|
||||||
|
state,
|
||||||
|
error_rate,
|
||||||
|
ambiguous,
|
||||||
|
norm_entropy,
|
||||||
|
)
|
||||||
|
|
||||||
|
class_probabilities = {
|
||||||
|
name: round(float(probs[i]) * 100, 2) for i, name in enumerate(class_names)
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
"class": predicted_class,
|
||||||
|
"confidence": round(max_prob * 100, 2),
|
||||||
|
"calibration": {
|
||||||
|
"raw_logits": [round(float(x), 6) for x in arr.tolist()],
|
||||||
|
"temperature": round(temperature, 4),
|
||||||
|
"base_temperature": cfg.temperature,
|
||||||
|
"mode": tier,
|
||||||
|
"clinical_suspicion": round(cfg.clinical_suspicion, 3),
|
||||||
|
"alpha_margin": cfg.alpha_margin,
|
||||||
|
"class_probabilities": class_probabilities,
|
||||||
|
"entropy": round(entropy, 4),
|
||||||
|
"normalized_entropy": round(norm_entropy, 4),
|
||||||
|
"ambiguous_set": ambiguous,
|
||||||
|
"decision_state": state,
|
||||||
|
"predicted_error_rate": round(error_rate, 4),
|
||||||
|
"risk_framing_vi": risk_vi,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def interpret_angle_logits(logits: Any, config: CalibrationConfig | None = None) -> dict[str, Any]:
|
||||||
|
return interpret_classification_logits(logits, ANGLE_CLASSES, config)
|
||||||
|
|
||||||
|
|
||||||
|
def interpret_inflammation_logits(logits: Any, config: CalibrationConfig | None = None) -> dict[str, Any]:
|
||||||
|
payload = interpret_classification_logits(logits, INFLAMMATION_CLASSES, config)
|
||||||
|
detected = payload["class"] == "inflammation"
|
||||||
|
payload["detected"] = detected
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def calibration_config_from_params(params: dict[str, Any] | None) -> CalibrationConfig:
|
||||||
|
if not params:
|
||||||
|
return CalibrationConfig()
|
||||||
|
calibration = params.get("calibration") or {}
|
||||||
|
return CalibrationConfig(
|
||||||
|
temperature=float(calibration.get("temperature", 1.0)),
|
||||||
|
mode=str(calibration.get("mode", "standard")),
|
||||||
|
clinical_suspicion=float(calibration.get("clinical_suspicion", 0.0)),
|
||||||
|
alpha_margin=float(calibration.get("alpha_margin", 0.05)),
|
||||||
|
ood_entropy_threshold=float(calibration.get("ood_entropy_threshold", 0.88)),
|
||||||
|
)
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
__all__ = ["calculate_thickness", "get_mask_bounding_box", "find_max_continuous_segment"]
|
||||||
|
import numpy as np
|
||||||
|
import cv2
|
||||||
|
|
||||||
|
SEGMENT_CLASSES_SUPRAPAT = {
|
||||||
|
0: "background", 1: "effusion", 2: "fat", 3: "fat-pat",
|
||||||
|
4: "femur", 5: "synovium", 6: "tendon"
|
||||||
|
}
|
||||||
|
SEGMENT_CLASSES_POST = {
|
||||||
|
0: "background", 1: "fat", 2: "tendon", 3: "muscle",
|
||||||
|
4: "femur", 5: "artery", 6: "baker's cyst"
|
||||||
|
}
|
||||||
|
PIXEL_TO_MM = 45.0 / 655.0
|
||||||
|
|
||||||
|
|
||||||
|
def get_mask_bounding_box(mask, dist_percent: float = 0.01):
|
||||||
|
if mask is None or np.sum(mask) == 0:
|
||||||
|
return None
|
||||||
|
mask_uint8 = mask.astype(np.uint8)
|
||||||
|
if np.max(mask_uint8) == 1:
|
||||||
|
mask_uint8 *= 255
|
||||||
|
img_width = mask_uint8.shape[1]
|
||||||
|
dist_threshold = img_width * dist_percent
|
||||||
|
kernel = np.ones((5, 5), np.uint8)
|
||||||
|
clean_mask = cv2.morphologyEx(mask_uint8, cv2.MORPH_OPEN, kernel)
|
||||||
|
contours, _ = cv2.findContours(clean_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
|
||||||
|
if not contours:
|
||||||
|
return None
|
||||||
|
contour_info = sorted(
|
||||||
|
[{"cnt": cnt, "area": cv2.contourArea(cnt)} for cnt in contours],
|
||||||
|
key=lambda x: x["area"], reverse=True,
|
||||||
|
)
|
||||||
|
main_block = contour_info[0]
|
||||||
|
max_area = main_block["area"]
|
||||||
|
if max_area < 50:
|
||||||
|
return None
|
||||||
|
main_mask = np.zeros_like(mask_uint8)
|
||||||
|
cv2.drawContours(main_mask, [main_block["cnt"]], -1, 255, -1)
|
||||||
|
dist_map = cv2.distanceTransform(255 - main_mask, cv2.DIST_L2, 3)
|
||||||
|
significant_contours = [main_block["cnt"]]
|
||||||
|
area_threshold = max_area / 4.0
|
||||||
|
for i in range(1, len(contour_info)):
|
||||||
|
other = contour_info[i]
|
||||||
|
other_mask = np.zeros_like(mask_uint8)
|
||||||
|
cv2.drawContours(other_mask, [other["cnt"]], -1, 255, -1)
|
||||||
|
min_dist = np.min(dist_map[other_mask > 0])
|
||||||
|
if other["area"] >= area_threshold or min_dist <= dist_threshold:
|
||||||
|
significant_contours.append(other["cnt"])
|
||||||
|
all_points = np.concatenate(significant_contours)
|
||||||
|
x, y, w, h = cv2.boundingRect(all_points)
|
||||||
|
return x, y, w, h
|
||||||
|
|
||||||
|
|
||||||
|
def find_max_continuous_segment(col_array):
|
||||||
|
padded = np.concatenate(([0], col_array, [0]))
|
||||||
|
diffs = np.diff(padded)
|
||||||
|
starts = np.where(diffs == 1)[0]
|
||||||
|
ends = np.where(diffs == -1)[0]
|
||||||
|
if len(starts) == 0:
|
||||||
|
return 0, -1, -1
|
||||||
|
lengths = ends - starts
|
||||||
|
max_idx = int(np.argmax(lengths))
|
||||||
|
max_len = int(lengths[max_idx])
|
||||||
|
return max_len, int(starts[max_idx]), int(ends[max_idx])
|
||||||
|
|
||||||
|
|
||||||
|
def calculate_thickness(masks: dict, image_size, measure_ids=None):
|
||||||
|
if measure_ids is None:
|
||||||
|
measure_ids = [1, 5]
|
||||||
|
width, height = image_size
|
||||||
|
mask_all_labels = np.zeros((height, width), dtype=np.uint8)
|
||||||
|
mask_measure = np.zeros((height, width), dtype=np.uint8)
|
||||||
|
has_any_label = False
|
||||||
|
if "fat-pat" in masks:
|
||||||
|
class_map = SEGMENT_CLASSES_SUPRAPAT
|
||||||
|
else:
|
||||||
|
class_map = SEGMENT_CLASSES_POST
|
||||||
|
for class_id, class_name in class_map.items():
|
||||||
|
if class_name not in masks or class_name == "background":
|
||||||
|
continue
|
||||||
|
mask = masks[class_name]
|
||||||
|
if np.sum(mask) > 0:
|
||||||
|
has_any_label = True
|
||||||
|
mask_all_labels = np.logical_or(mask_all_labels, mask).astype(np.uint8)
|
||||||
|
if class_id in measure_ids:
|
||||||
|
mask_measure = np.logical_or(mask_measure, mask).astype(np.uint8)
|
||||||
|
if not has_any_label or np.sum(mask_measure) == 0:
|
||||||
|
return None
|
||||||
|
bbox_all = get_mask_bounding_box(mask_all_labels)
|
||||||
|
if bbox_all is None:
|
||||||
|
return None
|
||||||
|
x_all, y_all, w_all, h_all = bbox_all
|
||||||
|
roi_start = x_all + (w_all // 3)
|
||||||
|
roi_end = x_all + (2 * w_all // 3)
|
||||||
|
roi_strip = mask_measure[:, roi_start:roi_end]
|
||||||
|
global_max_len_px = 0
|
||||||
|
best_x_rel = 0
|
||||||
|
best_y_start = 0
|
||||||
|
best_y_end = 0
|
||||||
|
for x in range(roi_strip.shape[1]):
|
||||||
|
col = roi_strip[:, x]
|
||||||
|
if not np.any(col):
|
||||||
|
continue
|
||||||
|
length, y_s, y_e = find_max_continuous_segment(col)
|
||||||
|
if length > global_max_len_px:
|
||||||
|
global_max_len_px = length
|
||||||
|
best_x_rel = x
|
||||||
|
best_y_start = y_s
|
||||||
|
best_y_end = y_e
|
||||||
|
if global_max_len_px == 0:
|
||||||
|
return None
|
||||||
|
thickness_mm = global_max_len_px * PIXEL_TO_MM
|
||||||
|
real_x = roi_start + best_x_rel
|
||||||
|
return {
|
||||||
|
"thickness_px": int(global_max_len_px),
|
||||||
|
"thickness_mm": float(round(thickness_mm, 2)),
|
||||||
|
"x": int(real_x),
|
||||||
|
"y_start": int(best_y_start),
|
||||||
|
"y_end": int(best_y_end),
|
||||||
|
"roi_start": int(roi_start),
|
||||||
|
"roi_end": int(roi_end),
|
||||||
|
"bbox": {"x": int(x_all), "y": int(y_all), "w": int(w_all), "h": int(h_all)},
|
||||||
|
}
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
__all__ = ["create_overlay"]
|
||||||
|
from PIL import Image, ImageDraw
|
||||||
|
import numpy as np
|
||||||
|
import cv2
|
||||||
|
|
||||||
|
COLOR_MAP_SUP = {
|
||||||
|
"background": [0, 0, 0],
|
||||||
|
"effusion": [255, 0, 0],
|
||||||
|
"fat": [255, 255, 0],
|
||||||
|
"fat-pat": [0, 255, 255],
|
||||||
|
"femur": [0, 255, 0],
|
||||||
|
"synovium": [255, 0, 255],
|
||||||
|
"tendon": [0, 0, 255],
|
||||||
|
}
|
||||||
|
|
||||||
|
COLOR_MAP_POST = {
|
||||||
|
"background": [0, 0, 0],
|
||||||
|
"baker's cyst": [255, 0, 0],
|
||||||
|
"fat": [255, 255, 0],
|
||||||
|
"muscle": [0, 255, 255],
|
||||||
|
"femur": [0, 255, 0],
|
||||||
|
"artery": [255, 0, 255],
|
||||||
|
"synovium": [255, 0, 255],
|
||||||
|
"tendon": [0, 0, 255],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def create_overlay(image_pil: Image.Image, masks: dict, measurement, angle_type: str = "sup") -> Image.Image:
|
||||||
|
if masks is None:
|
||||||
|
return image_pil
|
||||||
|
color_map = COLOR_MAP_SUP if angle_type == "sup" else COLOR_MAP_POST
|
||||||
|
img_array = np.array(image_pil)
|
||||||
|
overlay = img_array.copy()
|
||||||
|
for class_name, mask in masks.items():
|
||||||
|
if class_name in color_map and np.sum(mask) > 0:
|
||||||
|
color = color_map[class_name]
|
||||||
|
for i in range(3):
|
||||||
|
overlay[:, :, i] = np.where(
|
||||||
|
mask > 0,
|
||||||
|
(overlay[:, :, i] * 0.6 + color[i] * 0.4).astype(np.uint8),
|
||||||
|
overlay[:, :, i],
|
||||||
|
)
|
||||||
|
overlay_pil = Image.fromarray(overlay)
|
||||||
|
draw = ImageDraw.Draw(overlay_pil)
|
||||||
|
for class_name in ["effusion", "synovium"]:
|
||||||
|
mask = masks.get(class_name)
|
||||||
|
if mask is not None and np.sum(mask) > 0:
|
||||||
|
mask_uint8 = (mask * 255).astype(np.uint8)
|
||||||
|
contours, _ = cv2.findContours(mask_uint8, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
|
||||||
|
for contour in contours:
|
||||||
|
points = contour.reshape(-1, 2).tolist()
|
||||||
|
if len(points) > 2:
|
||||||
|
points = [(int(p[0]), int(p[1])) for p in points]
|
||||||
|
draw.line(points + [points[0]], fill=(255, 255, 255), width=3)
|
||||||
|
if measurement and angle_type == "sup":
|
||||||
|
x = measurement["x"]
|
||||||
|
y_start = measurement["y_start"]
|
||||||
|
y_end = measurement["y_end"]
|
||||||
|
thickness_mm = measurement["thickness_mm"]
|
||||||
|
roi_start = measurement["roi_start"]
|
||||||
|
roi_end = measurement["roi_end"]
|
||||||
|
bbox = measurement["bbox"]
|
||||||
|
draw.rectangle(
|
||||||
|
[bbox["x"], bbox["y"], bbox["x"] + bbox["w"], bbox["y"] + bbox["h"]],
|
||||||
|
outline=(0, 255, 0), width=3,
|
||||||
|
)
|
||||||
|
h = image_pil.size[1]
|
||||||
|
draw.line([(roi_start, 0), (roi_start, h)], fill=(0, 255, 255), width=2)
|
||||||
|
draw.line([(roi_end, 0), (roi_end, h)], fill=(0, 255, 255), width=2)
|
||||||
|
draw.line([(x, y_start), (x, y_end)], fill=(255, 0, 0), width=4)
|
||||||
|
radius = 4
|
||||||
|
draw.ellipse([x - radius, y_start - radius, x + radius, y_start + radius],
|
||||||
|
fill=(0, 255, 0), outline=(255, 255, 255), width=2)
|
||||||
|
draw.ellipse([x - radius, y_end - radius, x + radius, y_end + radius],
|
||||||
|
fill=(0, 255, 0), outline=(255, 255, 255), width=2)
|
||||||
|
text = f"{thickness_mm:.2f} mm"
|
||||||
|
try:
|
||||||
|
from PIL import ImageFont
|
||||||
|
font = ImageFont.load_default()
|
||||||
|
bbox_text = draw.textbbox((0, 0), text, font=font)
|
||||||
|
text_w = bbox_text[2] - bbox_text[0]
|
||||||
|
text_h = bbox_text[3] - bbox_text[1]
|
||||||
|
except Exception:
|
||||||
|
text_w, text_h = 100, 20
|
||||||
|
text_x = x + 8
|
||||||
|
text_y = y_start - text_h - 8
|
||||||
|
draw.rectangle([text_x - 2, text_y - 2, text_x + text_w + 2, text_y + text_h + 2], fill=(0, 0, 0))
|
||||||
|
draw.text((text_x, text_y), text, fill=(255, 0, 0))
|
||||||
|
return overlay_pil
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
__all__ = ["calculate_severity"]
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
SEVERITY_LEVELS = [
|
||||||
|
(15, 3, "Nặng", "#dc3545", "Dịch khớp dày, màng hoạt dịch tăng sinh rõ"),
|
||||||
|
(8, 2, "Trung bình", "#fd7e14", "Dịch khớp trung bình, màng hoạt dịch tăng sinh vừa"),
|
||||||
|
(3, 1, "Nhẹ", "#ffc107", "Dịch khớp mỏng, màng hoạt dịch tăng sinh nhẹ"),
|
||||||
|
(0, 0, "Rất nhẹ", "#28a745", "Lượng dịch và màng hoạt dịch trong giới hạn bình thường"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def calculate_severity(masks: dict, image_size) -> dict | None:
|
||||||
|
if not masks:
|
||||||
|
return None
|
||||||
|
width, height = image_size
|
||||||
|
total_pixels = width * height
|
||||||
|
effusion_mask = masks.get("effusion", np.zeros((height, width), dtype=np.uint8))
|
||||||
|
effusion_pixels = int(np.sum(effusion_mask))
|
||||||
|
effusion_ratio = (effusion_pixels / total_pixels) * 100
|
||||||
|
effusion_thickness = 0
|
||||||
|
if effusion_pixels > 0:
|
||||||
|
rows_with_effusion = np.any(effusion_mask > 0, axis=1)
|
||||||
|
if np.any(rows_with_effusion):
|
||||||
|
effusion_thickness = int(np.sum(rows_with_effusion))
|
||||||
|
synovium_mask = masks.get("synovium", np.zeros((height, width), dtype=np.uint8))
|
||||||
|
synovium_pixels = int(np.sum(synovium_mask))
|
||||||
|
synovium_ratio = (synovium_pixels / total_pixels) * 100
|
||||||
|
effusion_score = min(effusion_thickness / height * 100, 100)
|
||||||
|
synovium_score = synovium_ratio
|
||||||
|
combined_score = effusion_score * 0.6 + synovium_score * 0.4
|
||||||
|
for threshold, level, severity, color, description in SEVERITY_LEVELS:
|
||||||
|
if combined_score > threshold:
|
||||||
|
return {
|
||||||
|
"level": int(level),
|
||||||
|
"severity": severity,
|
||||||
|
"color": color,
|
||||||
|
"description": description,
|
||||||
|
"effusion": {
|
||||||
|
"pixels": effusion_pixels,
|
||||||
|
"ratio": float(round(effusion_ratio, 2)),
|
||||||
|
"thickness": effusion_thickness,
|
||||||
|
},
|
||||||
|
"synovium": {
|
||||||
|
"pixels": synovium_pixels,
|
||||||
|
"ratio": float(round(synovium_ratio, 2)),
|
||||||
|
},
|
||||||
|
"combined_score": float(round(combined_score, 2)),
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"level": 0,
|
||||||
|
"severity": "Rất nhẹ",
|
||||||
|
"color": "#28a745",
|
||||||
|
"description": "Lượng dịch và màng hoạt dịch trong giới hạn bình thường",
|
||||||
|
"effusion": {
|
||||||
|
"pixels": effusion_pixels,
|
||||||
|
"ratio": float(round(effusion_ratio, 2)),
|
||||||
|
"thickness": effusion_thickness,
|
||||||
|
},
|
||||||
|
"synovium": {
|
||||||
|
"pixels": synovium_pixels,
|
||||||
|
"ratio": float(round(synovium_ratio, 2)),
|
||||||
|
},
|
||||||
|
"combined_score": float(round(combined_score, 2)),
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
__all__ = ["apply_clahe"]
|
||||||
|
import cv2
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
|
||||||
|
def apply_clahe(image_pil: Image.Image, clip_limit: float = 2.0, tile_grid_size: tuple[int, int] = (8, 8)) -> Image.Image:
|
||||||
|
img_array = np.array(image_pil)
|
||||||
|
gray = cv2.cvtColor(img_array, cv2.COLOR_RGB2GRAY)
|
||||||
|
clahe = cv2.createCLAHE(clipLimit=clip_limit, tileGridSize=tile_grid_size)
|
||||||
|
enhanced_gray = clahe.apply(gray)
|
||||||
|
enhanced_rgb = cv2.cvtColor(enhanced_gray, cv2.COLOR_GRAY2RGB)
|
||||||
|
return Image.fromarray(enhanced_rgb)
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
__all__ = ["prepare_angle_tensor", "prepare_inflammation_tensor", "prepare_segmentation_tensor"]
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image
|
||||||
|
from .transforms import Resize, Normalize
|
||||||
|
|
||||||
|
ANGLE_TRANSFORM = Resize((224, 224))
|
||||||
|
ANGLE_NORMALIZE = Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
|
||||||
|
|
||||||
|
INFLAMMATION_TRANSFORM = Resize((224, 224))
|
||||||
|
INFLAMMATION_NORMALIZE = Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
|
||||||
|
|
||||||
|
SEGMENTATION_TRANSFORM = Resize((512, 512))
|
||||||
|
|
||||||
|
|
||||||
|
def _to_nchw(arr_hwc: np.ndarray) -> np.ndarray:
|
||||||
|
arr = arr_hwc.transpose(2, 0, 1)
|
||||||
|
return np.expand_dims(arr, axis=0)
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_angle_tensor(image_pil: Image.Image) -> np.ndarray:
|
||||||
|
img = ANGLE_TRANSFORM(image_pil)
|
||||||
|
arr = ANGLE_NORMALIZE(img)
|
||||||
|
return _to_nchw(arr)
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_inflammation_tensor(image_pil: Image.Image) -> np.ndarray:
|
||||||
|
img = INFLAMMATION_TRANSFORM(image_pil)
|
||||||
|
arr = INFLAMMATION_NORMALIZE(img)
|
||||||
|
return _to_nchw(arr)
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_segmentation_tensor(image_pil: Image.Image) -> np.ndarray:
|
||||||
|
img = SEGMENTATION_TRANSFORM(image_pil)
|
||||||
|
arr = np.asarray(img).astype(np.float32) / 255.0
|
||||||
|
return _to_nchw(arr)
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
__all__ = ["Resize", "Normalize"]
|
||||||
|
from PIL import Image
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
|
||||||
|
class Resize:
|
||||||
|
def __init__(self, size: tuple[int, int]):
|
||||||
|
self.size = size
|
||||||
|
|
||||||
|
def __call__(self, image: Image.Image) -> Image.Image:
|
||||||
|
return image.resize(self.size, Image.Resampling.BILINEAR)
|
||||||
|
|
||||||
|
|
||||||
|
class Normalize:
|
||||||
|
def __init__(self, mean: list[float], std: list[float]):
|
||||||
|
self.mean = np.array(mean, dtype=np.float32)
|
||||||
|
self.std = np.array(std, dtype=np.float32)
|
||||||
|
|
||||||
|
def __call__(self, image_pil: Image.Image) -> np.ndarray:
|
||||||
|
arr = np.asarray(image_pil).astype(np.float32) / 255.0
|
||||||
|
arr = (arr - self.mean) / self.std
|
||||||
|
return arr
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
from typing import Any, AsyncGenerator
|
||||||
|
import logging
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
from data.spec.schemas import (
|
||||||
|
HeatmapResult, RationaleResult, ChatResponse, DriftCheckResult,
|
||||||
|
EvidenceList, ActivationMeta, AnnotationArtifact, EscalationTicket,
|
||||||
|
GuardrailResult, CorrectionRecord,
|
||||||
|
)
|
||||||
|
from backend.implementation.adapters.llm_adapter import get_llm_adapter
|
||||||
|
from backend.implementation.adapters.bert_adapter import get_bert_adapter
|
||||||
|
from backend.implementation.adapters.redis_adapter import get_redis_client
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
llm_adapter = get_llm_adapter()
|
||||||
|
bert_adapter = get_bert_adapter()
|
||||||
|
redis_client = get_redis_client()
|
||||||
|
|
||||||
|
async def _verify_pre_egress(session_id: str, redaction_hash: str | None = None):
|
||||||
|
"""
|
||||||
|
Enforce NFR-16a Pre-Egress Checklist.
|
||||||
|
"""
|
||||||
|
# 1. Consent Verification
|
||||||
|
consent_key = f"consent:{session_id}"
|
||||||
|
if not redis_client.exists(consent_key):
|
||||||
|
logger.error(f"Consent missing for session {session_id}")
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail="User consent for cloud LLM egress is required."
|
||||||
|
)
|
||||||
|
|
||||||
|
# 2. Redaction Verification (if hash provided)
|
||||||
|
if redaction_hash:
|
||||||
|
# In real impl: Run Presidio on the prompt and compare hashes
|
||||||
|
# For now, we assume a simple check or stub
|
||||||
|
if redaction_hash == "FAIL_HASH":
|
||||||
|
logger.error(f"Redaction hash mismatch for session {session_id}")
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
|
detail="Redaction verification failed. PHI may be present."
|
||||||
|
)
|
||||||
|
|
||||||
|
# Note: Audit Log commit is handled via the LLM adapter's AuditCallbackHandler
|
||||||
|
# to ensure it happens exactly before the call.
|
||||||
|
|
||||||
|
async def gradcam(session_id: str) -> HeatmapResult:
|
||||||
|
raise NotImplementedError("Safety service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def rationale(session_id: str, redaction_hash: str | None = None) -> RationaleResult:
|
||||||
|
# Pre-egress check
|
||||||
|
await _verify_pre_egress(session_id, redaction_hash)
|
||||||
|
|
||||||
|
# 1. Fetch session context (simplified for stub)
|
||||||
|
context = {"grade": "moderate", "joint_site": "wrist"}
|
||||||
|
|
||||||
|
# 2. Construct prompt
|
||||||
|
prompt = f"Based on MOH guidelines, explain the synovitis grade {context['grade']} for {context['joint_site']}..."
|
||||||
|
|
||||||
|
# 3. Call LLM adapter
|
||||||
|
text = await llm_adapter.generate(prompt, session_id)
|
||||||
|
|
||||||
|
redis_client.set(f"consult_mode:{session_id}", "tier_3")
|
||||||
|
|
||||||
|
return RationaleResult(text=text)
|
||||||
|
|
||||||
|
|
||||||
|
async def circuit_break(session_id: str, flag: bool) -> None:
|
||||||
|
if flag:
|
||||||
|
logger.warning(f"Circuit breaker triggered for session {session_id}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def socratic_chat(session_id: str, prompt: str, redaction_hash: str | None = None) -> ChatResponse:
|
||||||
|
# Pre-egress check
|
||||||
|
await _verify_pre_egress(session_id, redaction_hash)
|
||||||
|
|
||||||
|
# 1. Retrieve conversation history (stub)
|
||||||
|
history = []
|
||||||
|
|
||||||
|
# 2. Construct prompt
|
||||||
|
full_prompt = f"History: {history}\nUser: {prompt}\nAssistant: "
|
||||||
|
|
||||||
|
# 3. Call LLM adapter
|
||||||
|
response_text = await llm_adapter.generate(full_prompt, session_id)
|
||||||
|
|
||||||
|
# 4. BERT Referee check (stub)
|
||||||
|
is_grounded = bert_adapter.referee_check(response_text, [])
|
||||||
|
if not is_grounded:
|
||||||
|
response_text = "I'm sorry, I couldn't verify this answer against the guidelines."
|
||||||
|
|
||||||
|
# Post-egress: update consult mode
|
||||||
|
redis_client.set(f"consult_mode:{session_id}", "tier_3")
|
||||||
|
|
||||||
|
return ChatResponse(response=response_text)
|
||||||
|
|
||||||
|
|
||||||
|
async def drift_check(session_id: str) -> DriftCheckResult:
|
||||||
|
res = bert_adapter.drift_check("mock clinical text")
|
||||||
|
return DriftCheckResult(score=res.score, is_drifted=res.is_drifted)
|
||||||
|
|
||||||
|
|
||||||
|
async def rag_evidence(session_id: str) -> EvidenceList:
|
||||||
|
raise NotImplementedError("Safety service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def activations(session_id: str, params: dict) -> ActivationMeta:
|
||||||
|
raise NotImplementedError("Safety service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def upload_artifact(session_id: str, file: Any) -> AnnotationArtifact:
|
||||||
|
raise NotImplementedError("Safety service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def ground_truth(session_id: str, label: dict) -> None:
|
||||||
|
raise NotImplementedError("Safety service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def escalate(session_id: str, reason: str) -> EscalationTicket:
|
||||||
|
raise NotImplementedError("Safety service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def morphology(session_id: str, annotation: dict) -> None:
|
||||||
|
raise NotImplementedError("Safety service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def guardrail_check(session_id: str, prompt: str, score: float) -> GuardrailResult:
|
||||||
|
res = bert_adapter.guardrail_check(prompt)
|
||||||
|
return GuardrailResult(verdict=res.verdict, reason=res.reason)
|
||||||
|
|
||||||
|
|
||||||
|
async def submit_correction(session_id: str, correction: dict) -> CorrectionRecord:
|
||||||
|
raise NotImplementedError("Safety correction service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def chat_stream(session_id: str, prompt: str, redaction_hash: str | None = None) -> AsyncGenerator[str, None]:
|
||||||
|
# Pre-egress check
|
||||||
|
await _verify_pre_egress(session_id, redaction_hash)
|
||||||
|
|
||||||
|
async for chunk in llm_adapter.stream_generate(prompt, session_id):
|
||||||
|
res = bert_adapter.guardrail_check(chunk)
|
||||||
|
if res.verdict == "MITIGATE":
|
||||||
|
yield "[Content Filtered]"
|
||||||
|
return
|
||||||
|
yield chunk
|
||||||
|
|
||||||
|
# Post-egress
|
||||||
|
redis_client.set(f"consult_mode:{session_id}", "tier_3")
|
||||||
|
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
from datetime import datetime
|
||||||
|
from data.spec.schemas import (
|
||||||
|
Session, SessionCreate, SessionDetail, SessionPatchReview,
|
||||||
|
FrameMetadata, PersistResult, ExportResult, ScrubResult,
|
||||||
|
)
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
async def create_session(user_id: str, patient_id: str, case_id: str | None = None) -> Session:
|
||||||
|
raise NotImplementedError("Session service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def get_session(session_id: str) -> SessionDetail:
|
||||||
|
raise NotImplementedError("Session service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def add_frame(session_id: str, file: Any, frame_number: int | None = None) -> FrameMetadata:
|
||||||
|
raise NotImplementedError("Session service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def patch_review(session_id: str, review: dict) -> Session:
|
||||||
|
raise NotImplementedError("Session service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def persist(session_id: str, review: dict) -> PersistResult:
|
||||||
|
raise NotImplementedError("Session service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def export_pdf(session_id: str, params: dict) -> ExportResult:
|
||||||
|
raise NotImplementedError("Session service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def scrub_validate(session_id: str, metadata: dict) -> ScrubResult:
|
||||||
|
raise NotImplementedError("Session service not yet implemented")
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
from data.spec.schemas import UserSettings, SettingsUpdate
|
||||||
|
|
||||||
|
|
||||||
|
async def get_settings(user_id: str) -> UserSettings:
|
||||||
|
raise NotImplementedError("Settings service not yet implemented")
|
||||||
|
|
||||||
|
|
||||||
|
async def update_settings(user_id: str, updates: dict) -> UserSettings:
|
||||||
|
raise NotImplementedError("Settings service not yet implemented")
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
from data.spec.schemas import AnomalyRecord
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
async def report_anomaly(session_id: str, data: dict) -> AnomalyRecord:
|
||||||
|
raise NotImplementedError("Telemetry service not yet implemented")
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
"""Triton batching helpers — aligned with config.pbtxt max_batch_size: 8."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
from collections.abc import Iterator, Sequence
|
||||||
|
from typing import TypeVar
|
||||||
|
|
||||||
|
T = TypeVar("T")
|
||||||
|
|
||||||
|
TRITON_MAX_BATCH_SIZE = int(os.getenv("TRITON_MAX_BATCH_SIZE", "8"))
|
||||||
|
|
||||||
|
|
||||||
|
def chunk_sequence(items: Sequence[T], batch_size: int | None = None) -> Iterator[list[T]]:
|
||||||
|
"""Split a sequence into chunks of at most ``batch_size`` (default: TRITON_MAX_BATCH_SIZE)."""
|
||||||
|
size = batch_size if batch_size is not None else TRITON_MAX_BATCH_SIZE
|
||||||
|
if size < 1:
|
||||||
|
raise ValueError(f"batch_size must be >= 1, got {size}")
|
||||||
|
for start in range(0, len(items), size):
|
||||||
|
yield list(items[start : start + size])
|
||||||
|
|
||||||
|
|
||||||
|
def batch_count(item_count: int, batch_size: int | None = None) -> int:
|
||||||
|
"""Number of Triton infer calls needed (e.g. 10 images -> 2 batches when size=8)."""
|
||||||
|
if item_count <= 0:
|
||||||
|
return 0
|
||||||
|
size = batch_size if batch_size is not None else TRITON_MAX_BATCH_SIZE
|
||||||
|
return (item_count + size - 1) // size
|
||||||
89
workspace/sprint_1_2/CODEBASE/backend/main.py
Normal file
89
workspace/sprint_1_2/CODEBASE/backend/main.py
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
from fastapi.exceptions import RequestValidationError
|
||||||
|
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||||
|
|
||||||
|
from backend.api import (
|
||||||
|
auth_api,
|
||||||
|
patient_api,
|
||||||
|
session_api,
|
||||||
|
analysis_api,
|
||||||
|
safety_api,
|
||||||
|
notification_api,
|
||||||
|
settings_api,
|
||||||
|
ingestion_api,
|
||||||
|
telemetry_api,
|
||||||
|
)
|
||||||
|
from backend.routers import cloud_orchestrate, cloud_consult, agent_tools
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(app: FastAPI):
|
||||||
|
logger.info("Starting medical imaging AI platform API")
|
||||||
|
yield
|
||||||
|
logger.info("Shutting down medical imaging AI platform API")
|
||||||
|
|
||||||
|
|
||||||
|
app = FastAPI(
|
||||||
|
title="Medical Imaging & AI Safety Platform",
|
||||||
|
description="Clinical diagnostic imaging platform with AI safety analysis",
|
||||||
|
version="0.1.0",
|
||||||
|
lifespan=lifespan,
|
||||||
|
)
|
||||||
|
|
||||||
|
app.add_middleware(
|
||||||
|
CORSMiddleware,
|
||||||
|
allow_origins=os.getenv(
|
||||||
|
"CORS_ORIGINS",
|
||||||
|
"http://localhost:3000,http://localhost:5173,http://localhost:5174,http://localhost:4173",
|
||||||
|
).split(","),
|
||||||
|
allow_credentials=True,
|
||||||
|
allow_methods=["*"],
|
||||||
|
allow_headers=["*"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.exception_handler(RequestValidationError)
|
||||||
|
async def validation_exception_handler(request, exc):
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from data.spec.schemas import ErrorResponse
|
||||||
|
return JSONResponse(
|
||||||
|
status_code=422,
|
||||||
|
content=ErrorResponse(detail=str(exc), code="VALIDATION_ERROR").model_dump(),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.exception_handler(StarletteHTTPException)
|
||||||
|
async def http_exception_handler(request, exc):
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from data.spec.schemas import ErrorResponse
|
||||||
|
content = ErrorResponse(detail=exc.detail, code="HTTP_ERROR").model_dump()
|
||||||
|
return JSONResponse(status_code=exc.status_code, content=content)
|
||||||
|
|
||||||
|
|
||||||
|
@app.exception_handler(NotImplementedError)
|
||||||
|
async def not_implemented_handler(request, exc):
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from data.spec.schemas import ErrorResponse
|
||||||
|
content = ErrorResponse(detail=str(exc), code="NOT_IMPLEMENTED").model_dump()
|
||||||
|
return JSONResponse(status_code=501, content=content)
|
||||||
|
|
||||||
|
|
||||||
|
app.include_router(cloud_orchestrate.router)
|
||||||
|
app.include_router(cloud_consult.router)
|
||||||
|
app.include_router(agent_tools.router)
|
||||||
|
|
||||||
|
app.include_router(auth_api.router)
|
||||||
|
app.include_router(patient_api.router)
|
||||||
|
app.include_router(session_api.router)
|
||||||
|
app.include_router(analysis_api.router)
|
||||||
|
app.include_router(safety_api.router)
|
||||||
|
app.include_router(notification_api.router)
|
||||||
|
app.include_router(settings_api.router)
|
||||||
|
app.include_router(ingestion_api.router)
|
||||||
|
app.include_router(telemetry_api.router)
|
||||||
118
workspace/sprint_1_2/CODEBASE/backend/routers/agent_tools.py
Normal file
118
workspace/sprint_1_2/CODEBASE/backend/routers/agent_tools.py
Normal file
@@ -0,0 +1,118 @@
|
|||||||
|
import logging
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from backend.services import agent_tools_service
|
||||||
|
from backend.services import embed_service
|
||||||
|
from data.spec.schemas import ErrorResponse
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["agent-tools"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login", auto_error=False)
|
||||||
|
|
||||||
|
|
||||||
|
class ExaSearchRequest(BaseModel):
|
||||||
|
query: str = Field(..., max_length=512)
|
||||||
|
type: str = "auto"
|
||||||
|
numResults: int = Field(default=10, ge=1, le=10)
|
||||||
|
includeDomains: list[str] | None = None
|
||||||
|
excludeDomains: list[str] | None = None
|
||||||
|
session_id: str
|
||||||
|
|
||||||
|
|
||||||
|
class SupabaseQueryRequest(BaseModel):
|
||||||
|
rpc: str
|
||||||
|
args: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
session_id: str
|
||||||
|
|
||||||
|
|
||||||
|
class EmbedRequest(BaseModel):
|
||||||
|
text: str = Field(..., max_length=8192)
|
||||||
|
task: str = "retrieval-query"
|
||||||
|
title: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_jwt_token_optional(token: str | None = Depends(oauth2_scheme)) -> str | None:
|
||||||
|
if not token:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/agent/tools/exa/search",
|
||||||
|
responses={
|
||||||
|
401: {"model": ErrorResponse},
|
||||||
|
422: {"model": ErrorResponse},
|
||||||
|
502: {"model": ErrorResponse},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
async def exa_search(
|
||||||
|
body: ExaSearchRequest,
|
||||||
|
user_id: str | None = Depends(_verify_jwt_token_optional),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await agent_tools_service.exa_search(body.model_dump())
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
except RuntimeError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=str(exc))
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
logger.exception("Exa upstream error")
|
||||||
|
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/embed",
|
||||||
|
responses={
|
||||||
|
401: {"model": ErrorResponse},
|
||||||
|
422: {"model": ErrorResponse},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
async def embed(
|
||||||
|
body: EmbedRequest,
|
||||||
|
user_id: str | None = Depends(_verify_jwt_token_optional),
|
||||||
|
):
|
||||||
|
task = body.task if body.task in {"retrieval-query", "retrieval-document"} else "retrieval-query"
|
||||||
|
try:
|
||||||
|
return await embed_service.embed_text(body.text, task=task, title=body.title)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/agent/tools/supabase/query",
|
||||||
|
responses={
|
||||||
|
401: {"model": ErrorResponse},
|
||||||
|
422: {"model": ErrorResponse},
|
||||||
|
501: {"model": ErrorResponse},
|
||||||
|
502: {"model": ErrorResponse},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
async def supabase_query(
|
||||||
|
body: SupabaseQueryRequest,
|
||||||
|
user_id: str | None = Depends(_verify_jwt_token_optional),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await agent_tools_service.supabase_query(body.model_dump())
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
except RuntimeError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=str(exc))
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
import logging
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, status, Body
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from pydantic import BaseModel
|
||||||
|
from data.spec.schemas import ErrorResponse
|
||||||
|
from backend.services.cloud_llm_gateway import route_medgemma_request
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["cloud-consult"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
|
||||||
|
class ConsultStreamRequest(BaseModel):
|
||||||
|
session_id: str
|
||||||
|
prompt: str
|
||||||
|
task_type: str = "clinical_deep_reasoning"
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/cloud-consult",
|
||||||
|
responses={401: {"model": ErrorResponse}, 403: {"model": ErrorResponse}, 502: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def cloud_consult(
|
||||||
|
payload: dict,
|
||||||
|
user_id: str = Depends(_verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await route_medgemma_request(payload, user_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
except PermissionError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/cloud-consult/stream",
|
||||||
|
responses={401: {"model": ErrorResponse}, 403: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def cloud_consult_stream(
|
||||||
|
body: ConsultStreamRequest,
|
||||||
|
user_id: str = Depends(_verify_jwt_token),
|
||||||
|
):
|
||||||
|
async def generate():
|
||||||
|
async for chunk in route_medgemma_request(
|
||||||
|
{
|
||||||
|
"session_id": body.session_id,
|
||||||
|
"prompt": body.prompt,
|
||||||
|
"task_type": body.task_type,
|
||||||
|
"stream": True,
|
||||||
|
},
|
||||||
|
user_id,
|
||||||
|
):
|
||||||
|
yield chunk
|
||||||
|
|
||||||
|
return StreamingResponse(
|
||||||
|
generate(),
|
||||||
|
media_type="text/event-stream",
|
||||||
|
headers={
|
||||||
|
"Cache-Control": "no-cache",
|
||||||
|
"X-Accel-Buffering": "no",
|
||||||
|
"Connection": "keep-alive",
|
||||||
|
},
|
||||||
|
)
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
import logging
|
||||||
|
import httpx
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, status, Body
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
from fastapi.security import OAuth2PasswordBearer
|
||||||
|
from data.spec.schemas import ErrorResponse
|
||||||
|
from backend.services.cloud_llm_gateway import route_gemini_request
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1", tags=["cloud-orchestrate"])
|
||||||
|
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
|
||||||
|
try:
|
||||||
|
from backend.api.auth_api import verify_jwt_token as _verify
|
||||||
|
return await _verify(token)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid or expired token",
|
||||||
|
headers={"WWW-Authenticate": "Bearer"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post(
|
||||||
|
"/cloud-orchestrate",
|
||||||
|
responses={401: {"model": ErrorResponse}, 403: {"model": ErrorResponse}, 502: {"model": ErrorResponse}},
|
||||||
|
)
|
||||||
|
async def cloud_orchestrate(
|
||||||
|
payload: dict,
|
||||||
|
user_id: str = Depends(_verify_jwt_token),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
return await route_gemini_request(payload, user_id)
|
||||||
|
except NotImplementedError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
|
||||||
|
except PermissionError as exc:
|
||||||
|
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(exc))
|
||||||
223
workspace/sprint_1_2/CODEBASE/backend/routers/cv_inference.py
Normal file
223
workspace/sprint_1_2/CODEBASE/backend/routers/cv_inference.py
Normal file
@@ -0,0 +1,223 @@
|
|||||||
|
"""HTTP routes for spec-compliant CV inference (CLAHE → angle → inflammation → seg)."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
|
||||||
|
import requests
|
||||||
|
from fastapi import APIRouter, File, Form, HTTPException, UploadFile
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
from backend.implementation.adapters.triton_adapter import TritonAdapter
|
||||||
|
from backend.implementation.config import get_model_name, get_segmentation_model
|
||||||
|
from backend.implementation.postprocessing.calibration import CalibrationConfig, calibration_config_from_params
|
||||||
|
from backend.implementation.triton_batch import TRITON_MAX_BATCH_SIZE
|
||||||
|
from backend.services import cv_result_cache
|
||||||
|
from backend.services import triton_runtime_service as triton_runtime
|
||||||
|
from backend.services.cv_inference_service import CvBatchResult, CvInferenceOptions, run_batch, run_single
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/test", tags=["cv-inference"])
|
||||||
|
|
||||||
|
LEGACY_DEPRECATION_DETAIL = (
|
||||||
|
"This endpoint is deprecated. Use POST /api/test/analyze or POST /api/test/analyze/batch "
|
||||||
|
"for the spec-compliant CV pipeline."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_image_upload(content_type: str | None, filename: str | None) -> bool:
|
||||||
|
if content_type and content_type.startswith("image/"):
|
||||||
|
return True
|
||||||
|
if content_type in (None, "", "application/octet-stream", "binary/octet-stream"):
|
||||||
|
name = (filename or "").lower()
|
||||||
|
return name.endswith((".png", ".jpg", ".jpeg", ".webp", ".bmp", ".gif"))
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_calibration_form(calibration_json: str | None) -> CalibrationConfig:
|
||||||
|
if not calibration_json:
|
||||||
|
return CalibrationConfig()
|
||||||
|
try:
|
||||||
|
data = json.loads(calibration_json)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return CalibrationConfig()
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
return CalibrationConfig()
|
||||||
|
return calibration_config_from_params({"calibration": data})
|
||||||
|
|
||||||
|
|
||||||
|
def _default_model_versions() -> dict[str, str] | None:
|
||||||
|
versions: dict[str, str] = {}
|
||||||
|
if angle := os.getenv("ANGLE_MODEL"):
|
||||||
|
versions["angle"] = angle
|
||||||
|
elif os.getenv("CV_USE_CONFIG_ANGLE_MODEL", "").lower() not in {"1", "true", "yes"}:
|
||||||
|
# Match legacy test proxy default for PoC clinical accuracy
|
||||||
|
versions["angle"] = "angle_classify_resnet50"
|
||||||
|
if inflam := os.getenv("INFLAMMATION_MODEL"):
|
||||||
|
versions["inflammation"] = inflam
|
||||||
|
if seg := os.getenv("SEGMENT_MODEL"):
|
||||||
|
versions["segmentation_sup"] = seg
|
||||||
|
versions["segmentation_post"] = seg
|
||||||
|
return versions or None
|
||||||
|
|
||||||
|
|
||||||
|
def _build_options(
|
||||||
|
calibration: CalibrationConfig,
|
||||||
|
*,
|
||||||
|
use_cache: bool = True,
|
||||||
|
) -> CvInferenceOptions:
|
||||||
|
return CvInferenceOptions(
|
||||||
|
calibration=calibration,
|
||||||
|
model_versions=_default_model_versions(),
|
||||||
|
use_cache=use_cache,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _load_upload_image(upload: UploadFile) -> Image.Image:
|
||||||
|
if not _is_image_upload(upload.content_type, upload.filename):
|
||||||
|
raise HTTPException(status_code=400, detail=f"Expected images, got {upload.filename}")
|
||||||
|
try:
|
||||||
|
raw = await upload.read()
|
||||||
|
return Image.open(io.BytesIO(raw)).convert("RGB")
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=400, detail=f"Invalid image {upload.filename}: {exc}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _triton_http_error_detail(exc: requests.HTTPError, operation: str) -> str:
|
||||||
|
status = exc.response.status_code if exc.response is not None else 503
|
||||||
|
detail = (
|
||||||
|
f"{operation} failed ({status}). "
|
||||||
|
"Modal server may be cold-starting — retry in a few seconds."
|
||||||
|
)
|
||||||
|
if exc.response is not None and exc.response.text:
|
||||||
|
detail = f"{detail} Server: {exc.response.text[:300]}"
|
||||||
|
return detail
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/health")
|
||||||
|
async def cv_inference_health():
|
||||||
|
angle_model = get_model_name("angle", _default_model_versions())
|
||||||
|
inflam_model = get_model_name("inflammation", _default_model_versions())
|
||||||
|
seg_model = get_segmentation_model("sup-up-long", _default_model_versions())
|
||||||
|
triton_endpoint = triton_runtime.get_triton_endpoint()
|
||||||
|
try:
|
||||||
|
adapter = TritonAdapter(endpoint_url=triton_endpoint, timeout=triton_runtime.TRITON_INFER_TIMEOUT)
|
||||||
|
angle_ready = await adapter.model_ready(angle_model)
|
||||||
|
inflam_ready = await adapter.model_ready(inflam_model)
|
||||||
|
seg_ready = await adapter.model_ready(seg_model)
|
||||||
|
status = "ok" if angle_ready and inflam_ready and seg_ready else "degraded"
|
||||||
|
cache = cv_result_cache.cache_stats()
|
||||||
|
return {
|
||||||
|
"status": status,
|
||||||
|
"service": "cv-inference",
|
||||||
|
"triton": triton_endpoint,
|
||||||
|
"angle_model": angle_model,
|
||||||
|
"angle_ready": angle_ready,
|
||||||
|
"inflammation_model": inflam_model,
|
||||||
|
"inflammation_ready": inflam_ready,
|
||||||
|
"segmentation_model": seg_model,
|
||||||
|
"segmentation_ready": seg_ready,
|
||||||
|
"triton_max_batch_size": TRITON_MAX_BATCH_SIZE,
|
||||||
|
"triton_infer_timeout": triton_runtime.TRITON_INFER_TIMEOUT,
|
||||||
|
"triton_infer_retries": triton_runtime.TRITON_INFER_RETRIES,
|
||||||
|
"triton_use_batch_infer": triton_runtime.TRITON_USE_BATCH_INFER,
|
||||||
|
**cache,
|
||||||
|
}
|
||||||
|
except Exception as exc:
|
||||||
|
logger.exception("CV inference health check failed")
|
||||||
|
return JSONResponse(
|
||||||
|
status_code=503,
|
||||||
|
content={"status": "error", "service": "cv-inference", "detail": str(exc), "triton": triton_endpoint},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/analyze")
|
||||||
|
async def analyze_upload(
|
||||||
|
image: UploadFile = File(...),
|
||||||
|
calibration: str | None = Form(default=None),
|
||||||
|
):
|
||||||
|
"""Spec-compliant CV pipeline: CLAHE → angle → inflammation → conditional segmentation."""
|
||||||
|
image_pil = await _load_upload_image(image)
|
||||||
|
options = _build_options(_parse_calibration_form(calibration), use_cache=False)
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = await run_single(image_pil, frame_id=None, options=options)
|
||||||
|
return JSONResponse(result)
|
||||||
|
except requests.HTTPError as exc:
|
||||||
|
logger.exception("Analyze pipeline failed (Triton HTTP)")
|
||||||
|
raise HTTPException(status_code=503, detail=_triton_http_error_detail(exc, "Triton analyze pipeline")) from exc
|
||||||
|
except (requests.ConnectionError, requests.Timeout) as exc:
|
||||||
|
logger.exception("Analyze pipeline failed (Triton network)")
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=503,
|
||||||
|
detail=f"Triton unreachable: {exc}. Check TRITON_ENDPOINT and Modal deployment.",
|
||||||
|
) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
logger.exception("Analyze pipeline failed")
|
||||||
|
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/analyze/batch")
|
||||||
|
async def analyze_batch_upload(
|
||||||
|
images: list[UploadFile] = File(...),
|
||||||
|
frame_ids: str = Form(...),
|
||||||
|
calibration: str | None = Form(default=None),
|
||||||
|
):
|
||||||
|
"""Spec-compliant CV batch — one full pipeline per frame (angle-first, gated segmentation)."""
|
||||||
|
if not images:
|
||||||
|
raise HTTPException(status_code=400, detail="At least one image is required")
|
||||||
|
|
||||||
|
try:
|
||||||
|
id_list = json.loads(frame_ids)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise HTTPException(status_code=400, detail="frame_ids must be a JSON array of strings") from exc
|
||||||
|
|
||||||
|
if not isinstance(id_list, list) or not all(isinstance(x, str) for x in id_list):
|
||||||
|
raise HTTPException(status_code=400, detail="frame_ids must be a JSON array of strings")
|
||||||
|
if len(id_list) != len(images):
|
||||||
|
raise HTTPException(status_code=400, detail="frame_ids length must match images count")
|
||||||
|
|
||||||
|
image_pils: list[Image.Image] = []
|
||||||
|
for upload in images:
|
||||||
|
image_pils.append(await _load_upload_image(upload))
|
||||||
|
|
||||||
|
options = _build_options(_parse_calibration_form(calibration))
|
||||||
|
|
||||||
|
try:
|
||||||
|
batch: CvBatchResult = await run_batch(image_pils, id_list, options=options)
|
||||||
|
cache = cv_result_cache.cache_stats()
|
||||||
|
return JSONResponse({
|
||||||
|
"success": True,
|
||||||
|
"image_count": len(batch.results),
|
||||||
|
"pipeline": "spec-cv-v1",
|
||||||
|
"triton_infer_calls": batch.triton_infer_calls,
|
||||||
|
"triton_infer_mode": batch.triton_infer_modes,
|
||||||
|
"pipeline_version": cache["pipeline_version"],
|
||||||
|
"results": batch.results,
|
||||||
|
})
|
||||||
|
except requests.HTTPError as exc:
|
||||||
|
logger.exception("Analyze batch pipeline failed (Triton HTTP)")
|
||||||
|
raise HTTPException(status_code=503, detail=_triton_http_error_detail(exc, "Triton analyze batch")) from exc
|
||||||
|
except (requests.ConnectionError, requests.Timeout) as exc:
|
||||||
|
logger.exception("Analyze batch pipeline failed (Triton network)")
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=503,
|
||||||
|
detail=f"Triton unreachable: {exc}. Check TRITON_ENDPOINT and Modal deployment.",
|
||||||
|
) from exc
|
||||||
|
except Exception as exc:
|
||||||
|
logger.exception("Analyze batch pipeline failed")
|
||||||
|
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/segment")
|
||||||
|
@router.post("/segment/batch")
|
||||||
|
@router.post("/angle")
|
||||||
|
@router.post("/angle/batch")
|
||||||
|
@router.post("/inflammation")
|
||||||
|
@router.post("/inflammation/batch")
|
||||||
|
async def legacy_cv_endpoints_deprecated():
|
||||||
|
raise HTTPException(status_code=410, detail=LEGACY_DEPRECATION_DETAIL)
|
||||||
@@ -0,0 +1,184 @@
|
|||||||
|
"""Agent tool BFF services — Exa search and Supabase knowledge queries."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
EXA_SEARCH_URL = "https://api.exa.ai/search"
|
||||||
|
EXA_API_KEY = os.getenv("EXA_API_KEY", "").strip()
|
||||||
|
|
||||||
|
SUPABASE_URL = os.getenv("SUPABASE_URL", "").rstrip("/")
|
||||||
|
SUPABASE_SERVICE_ROLE_KEY = os.getenv("SUPABASE_SERVICE_ROLE_KEY", "").strip()
|
||||||
|
|
||||||
|
ALLOWED_EXA_TYPES = frozenset(
|
||||||
|
{"auto", "fast", "instant", "deep-lite", "deep", "deep-reasoning"}
|
||||||
|
)
|
||||||
|
ALLOWED_SUPABASE_RPC = frozenset({"match_semantic_chunks", "get_corpus_citation"})
|
||||||
|
|
||||||
|
|
||||||
|
def _audit(event: str, session_id: str, payload: dict[str, Any]) -> None:
|
||||||
|
query_hash = payload.get("query_hash")
|
||||||
|
logger.info(
|
||||||
|
"[AUDIT] event=%s session=%s query_hash=%s payload_keys=%s",
|
||||||
|
event,
|
||||||
|
session_id,
|
||||||
|
query_hash,
|
||||||
|
list(payload.keys()),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _query_hash(text: str) -> str:
|
||||||
|
return hashlib.sha256(text.encode("utf-8")).hexdigest()[:16]
|
||||||
|
|
||||||
|
|
||||||
|
async def exa_search(payload: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
session_id = str(payload.get("session_id", ""))
|
||||||
|
query = str(payload.get("query", "")).strip()
|
||||||
|
if not query:
|
||||||
|
raise ValueError("query is required")
|
||||||
|
if len(query) > 512:
|
||||||
|
raise ValueError("query exceeds 512 characters")
|
||||||
|
|
||||||
|
search_type = str(payload.get("type", "auto"))
|
||||||
|
if search_type not in ALLOWED_EXA_TYPES:
|
||||||
|
raise ValueError(f"unsupported Exa type: {search_type}")
|
||||||
|
|
||||||
|
num_results = int(payload.get("numResults", 10))
|
||||||
|
num_results = max(1, min(num_results, 10))
|
||||||
|
|
||||||
|
if not EXA_API_KEY:
|
||||||
|
raise RuntimeError("EXA_API_KEY is not configured on the backend")
|
||||||
|
|
||||||
|
body: dict[str, Any] = {
|
||||||
|
"query": query,
|
||||||
|
"type": search_type,
|
||||||
|
"numResults": num_results,
|
||||||
|
"contents": {"highlights": True},
|
||||||
|
}
|
||||||
|
|
||||||
|
include_domains = payload.get("includeDomains")
|
||||||
|
exclude_domains = payload.get("excludeDomains")
|
||||||
|
if include_domains:
|
||||||
|
body["includeDomains"] = include_domains
|
||||||
|
if exclude_domains:
|
||||||
|
body["excludeDomains"] = exclude_domains
|
||||||
|
|
||||||
|
_audit(
|
||||||
|
"exa_search",
|
||||||
|
session_id,
|
||||||
|
{"query_hash": _query_hash(query), "type": search_type, "numResults": num_results},
|
||||||
|
)
|
||||||
|
|
||||||
|
headers = {"x-api-key": EXA_API_KEY, "Content-Type": "application/json"}
|
||||||
|
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||||
|
response = await client.post(EXA_SEARCH_URL, headers=headers, json=body)
|
||||||
|
response.raise_for_status()
|
||||||
|
data = response.json()
|
||||||
|
|
||||||
|
hits = []
|
||||||
|
for index, row in enumerate(data.get("results", [])):
|
||||||
|
hits.append(
|
||||||
|
{
|
||||||
|
"id": row.get("id") or f"exa-{index}",
|
||||||
|
"url": row.get("url") or "",
|
||||||
|
"title": row.get("title") or row.get("url") or "Untitled",
|
||||||
|
"highlights": row.get("highlights") or [],
|
||||||
|
"publishedDate": row.get("publishedDate"),
|
||||||
|
"score": row.get("score"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
return {"hits": hits, "requestId": data.get("requestId")}
|
||||||
|
|
||||||
|
|
||||||
|
async def supabase_query(payload: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
session_id = str(payload.get("session_id", ""))
|
||||||
|
rpc = str(payload.get("rpc", ""))
|
||||||
|
if rpc not in ALLOWED_SUPABASE_RPC:
|
||||||
|
raise ValueError(f"rpc not allowlisted: {rpc}")
|
||||||
|
|
||||||
|
if not SUPABASE_URL or not SUPABASE_SERVICE_ROLE_KEY:
|
||||||
|
raise RuntimeError("SUPABASE_URL and SUPABASE_SERVICE_ROLE_KEY must be configured")
|
||||||
|
|
||||||
|
args = payload.get("args") or {}
|
||||||
|
if rpc == "match_semantic_chunks":
|
||||||
|
query_text = str(args.get("query_text", "")).strip()
|
||||||
|
if not query_text:
|
||||||
|
raise ValueError("args.query_text is required for match_semantic_chunks")
|
||||||
|
|
||||||
|
_audit(
|
||||||
|
"supabase_query",
|
||||||
|
session_id,
|
||||||
|
{"query_hash": _query_hash(query_text), "rpc": rpc},
|
||||||
|
)
|
||||||
|
|
||||||
|
embedding = await _embed_query_text(query_text)
|
||||||
|
rpc_body = {
|
||||||
|
"query_embedding": embedding,
|
||||||
|
"match_count": int(args.get("match_count", 5)),
|
||||||
|
"filter_book_ids": args.get("filter_book_ids"),
|
||||||
|
"filter_edition_ids": args.get("filter_edition_ids"),
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
_audit("supabase_query", session_id, {"rpc": rpc})
|
||||||
|
rpc_body = args
|
||||||
|
|
||||||
|
url = f"{SUPABASE_URL}/rest/v1/rpc/{rpc}"
|
||||||
|
headers = {
|
||||||
|
"apikey": SUPABASE_SERVICE_ROLE_KEY,
|
||||||
|
"Authorization": f"Bearer {SUPABASE_SERVICE_ROLE_KEY}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Accept-Profile": "knowledge",
|
||||||
|
"Content-Profile": "knowledge",
|
||||||
|
}
|
||||||
|
|
||||||
|
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||||
|
response = await client.post(url, headers=headers, json=rpc_body)
|
||||||
|
if response.status_code >= 400:
|
||||||
|
detail = response.text
|
||||||
|
try:
|
||||||
|
detail = json.dumps(response.json())
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
raise RuntimeError(f"Supabase RPC failed ({response.status_code}): {detail}")
|
||||||
|
rows = response.json()
|
||||||
|
|
||||||
|
normalized_rows = []
|
||||||
|
for row in rows if isinstance(rows, list) else []:
|
||||||
|
normalized_rows.append(
|
||||||
|
{
|
||||||
|
"chunk_id": str(row.get("chunk_id", "")),
|
||||||
|
"content": row.get("content") or "",
|
||||||
|
"book_id": row.get("book_id") or "",
|
||||||
|
"parent_title": row.get("parent_title"),
|
||||||
|
"page_start": row.get("page_start"),
|
||||||
|
"page_end": row.get("page_end"),
|
||||||
|
"similarity": row.get("similarity"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
return {"rpc": rpc, "rows": normalized_rows}
|
||||||
|
|
||||||
|
|
||||||
|
async def _embed_query_text(query_text: str) -> list[float]:
|
||||||
|
"""Compute 768-d EmbeddingGemma vector for Supabase RPC.
|
||||||
|
|
||||||
|
PoC: returns NotImplemented until Triton/on-prem embedder is wired.
|
||||||
|
Set EMBED_QUERY_MOCK=1 to return a zero vector for integration testing only.
|
||||||
|
"""
|
||||||
|
if os.getenv("EMBED_QUERY_MOCK") == "1":
|
||||||
|
logger.warning("Using EMBED_QUERY_MOCK zero vector — not for production search quality")
|
||||||
|
return [0.0] * 768
|
||||||
|
|
||||||
|
raise NotImplementedError(
|
||||||
|
"Server-side query embedding is not wired yet. "
|
||||||
|
"Configure Triton EmbeddingGemma or set EMBED_QUERY_MOCK=1 for integration tests."
|
||||||
|
)
|
||||||
@@ -0,0 +1,203 @@
|
|||||||
|
import logging
|
||||||
|
import httpx
|
||||||
|
import json
|
||||||
|
from typing import AsyncGenerator
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from backend.implementation.adapters.redis_adapter import get_redis_client
|
||||||
|
from backend.implementation.adapters.llm_adapter import get_llm_adapter, AuditCallbackHandler
|
||||||
|
from backend.implementation.config import (
|
||||||
|
MODAL_MEDGEMMA_ENDPOINT,
|
||||||
|
VERTEX_AI_GEMINI_ENDPOINT,
|
||||||
|
GCP_ACCESS_TOKEN,
|
||||||
|
PROJECT_ID,
|
||||||
|
LOCATION,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
redis_client = get_redis_client()
|
||||||
|
llm_adapter = get_llm_adapter()
|
||||||
|
|
||||||
|
|
||||||
|
def _set_consult_mode(session_id: str, mode: str):
|
||||||
|
redis_client.set(f"consult_mode:{session_id}", mode, ex=7200)
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_consent(session_id: str) -> bool:
|
||||||
|
consent_key = f"consent:{session_id}"
|
||||||
|
return bool(await asyncio.to_thread(redis_client.exists, consent_key))
|
||||||
|
|
||||||
|
|
||||||
|
async def _verify_session_ownership(session_id: str, user_id: str) -> bool:
|
||||||
|
owner_key = f"session_owner:{session_id}"
|
||||||
|
owner_id = await asyncio.to_thread(redis_client.get, owner_key)
|
||||||
|
if not owner_id:
|
||||||
|
return False
|
||||||
|
return owner_id == user_id
|
||||||
|
|
||||||
|
|
||||||
|
def _audit_event(session_id: str, event_type: str, payload: dict):
|
||||||
|
key = f"audit:{session_id}:{event_type}"
|
||||||
|
redis_client.set(key, json.dumps(payload), ex=86400)
|
||||||
|
logger.info(
|
||||||
|
"[AUDIT] event=%s session=%s payload=%s",
|
||||||
|
event_type,
|
||||||
|
session_id,
|
||||||
|
payload,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def route_gemini_request(payload: dict, user_id: str) -> dict:
|
||||||
|
session_id = payload.get("session_id", "")
|
||||||
|
task_type = payload.get("task_type", "orchestration")
|
||||||
|
prompt = payload.get("prompt", "")
|
||||||
|
redaction_hash = payload.get("redaction_hash")
|
||||||
|
|
||||||
|
if not await _verify_consent(session_id):
|
||||||
|
raise PermissionError("User consent for cloud LLM egress is required.")
|
||||||
|
|
||||||
|
if not await _verify_session_ownership(session_id, user_id):
|
||||||
|
raise PermissionError("You do not own this session.")
|
||||||
|
|
||||||
|
_audit_event(session_id, "egress_consent_gemini", {
|
||||||
|
"user_id": user_id,
|
||||||
|
"task_type": task_type,
|
||||||
|
"redaction_hash": redaction_hash,
|
||||||
|
"ts": datetime.utcnow().isoformat(),
|
||||||
|
})
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {GCP_ACCESS_TOKEN}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
}
|
||||||
|
|
||||||
|
vertex_payload = {
|
||||||
|
"contents": [{"parts": [{"text": prompt}]}],
|
||||||
|
"generationConfig": {
|
||||||
|
"temperature": 0.2,
|
||||||
|
"topP": 0.8,
|
||||||
|
"topK": 40,
|
||||||
|
"maxOutputTokens": 1024,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
async with httpx.AsyncClient(timeout=22.0) as client:
|
||||||
|
response = await client.post(
|
||||||
|
VERTEX_AI_GEMINI_ENDPOINT,
|
||||||
|
headers=headers,
|
||||||
|
json=vertex_payload,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
result = response.json()
|
||||||
|
|
||||||
|
_audit_event(session_id, "egress_response_gemini", {
|
||||||
|
"task_type": task_type,
|
||||||
|
"status": "success",
|
||||||
|
"ts": datetime.utcnow().isoformat(),
|
||||||
|
})
|
||||||
|
_set_consult_mode(session_id, "tier_2")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"text": result.get("candidates", [{}])[0].get("content", {}).get("parts", [{}])[0].get("text", ""),
|
||||||
|
"tier": "gemini",
|
||||||
|
"task_type": task_type,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
async def route_medgemma_request(payload: dict, user_id: str) -> dict | AsyncGenerator[str, None]:
|
||||||
|
session_id = payload.get("session_id", "")
|
||||||
|
task_type = payload.get("task_type", "clinical_deep_reasoning")
|
||||||
|
prompt = payload.get("prompt", "")
|
||||||
|
stream = payload.get("stream", False)
|
||||||
|
redaction_hash = payload.get("redaction_hash")
|
||||||
|
|
||||||
|
if not await _verify_consent(session_id):
|
||||||
|
raise PermissionError("User consent for cloud LLM egress is required.")
|
||||||
|
|
||||||
|
if not await _verify_session_ownership(session_id, user_id):
|
||||||
|
raise PermissionError("You do not own this session.")
|
||||||
|
|
||||||
|
_audit_event(session_id, "egress_consent_medgemma", {
|
||||||
|
"user_id": user_id,
|
||||||
|
"task_type": task_type,
|
||||||
|
"redaction_hash": redaction_hash,
|
||||||
|
"ts": datetime.utcnow().isoformat(),
|
||||||
|
})
|
||||||
|
|
||||||
|
modal_payload = {
|
||||||
|
"model": payload.get("model", "medgemma:4b"),
|
||||||
|
"prompt": prompt,
|
||||||
|
"stream": stream,
|
||||||
|
"options": {
|
||||||
|
"temperature": 0.1,
|
||||||
|
"top_p": 0.8,
|
||||||
|
"top_k": 40,
|
||||||
|
"num_predict": 2048,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
}
|
||||||
|
|
||||||
|
if stream:
|
||||||
|
return _stream_medgemma(session_id, modal_payload, headers, task_type)
|
||||||
|
|
||||||
|
async with httpx.AsyncClient(timeout=22.0) as client:
|
||||||
|
response = await client.post(
|
||||||
|
f"{MODAL_MEDGEMMA_ENDPOINT}/api/generate",
|
||||||
|
headers=headers,
|
||||||
|
json=modal_payload,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
result = response.json()
|
||||||
|
|
||||||
|
_audit_event(session_id, "egress_response_medgemma", {
|
||||||
|
"task_type": task_type,
|
||||||
|
"status": "success",
|
||||||
|
"ts": datetime.utcnow().isoformat(),
|
||||||
|
})
|
||||||
|
_set_consult_mode(session_id, "tier_3")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"text": result.get("response", ""),
|
||||||
|
"tier": "medgemma",
|
||||||
|
"task_type": task_type,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
async def _stream_medgemma(
|
||||||
|
session_id: str,
|
||||||
|
modal_payload: dict,
|
||||||
|
headers: dict,
|
||||||
|
task_type: str,
|
||||||
|
) -> AsyncGenerator[str, None]:
|
||||||
|
async with httpx.AsyncClient(timeout=22.0) as client:
|
||||||
|
async with client.stream(
|
||||||
|
"POST",
|
||||||
|
f"{MODAL_MEDGEMMA_ENDPOINT}/api/generate",
|
||||||
|
headers=headers,
|
||||||
|
json=modal_payload,
|
||||||
|
) as response:
|
||||||
|
response.raise_for_status()
|
||||||
|
async for line in response.aiter_lines():
|
||||||
|
if not line.startswith("data:"):
|
||||||
|
continue
|
||||||
|
data_str = line[len("data:"):].strip()
|
||||||
|
if not data_str:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
data = json.loads(data_str)
|
||||||
|
chunk = data.get("response", "")
|
||||||
|
if chunk:
|
||||||
|
yield chunk
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
|
||||||
|
_audit_event(session_id, "egress_response_medgemma_stream", {
|
||||||
|
"task_type": task_type,
|
||||||
|
"status": "success",
|
||||||
|
"ts": datetime.utcnow().isoformat(),
|
||||||
|
})
|
||||||
|
_set_consult_mode(session_id, "tier_3")
|
||||||
@@ -0,0 +1,336 @@
|
|||||||
|
"""Spec-compliant CV inference orchestration — Sprint 1–2 architecture §7."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import base64
|
||||||
|
import io
|
||||||
|
import logging
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import cv2
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
from backend.implementation.config import get_angle_type, get_model_name, get_segmentation_model
|
||||||
|
from backend.implementation.pipeline.cv_spec_pipeline import (
|
||||||
|
BRANCH_ANGLE_CLASSES,
|
||||||
|
build_segmentation_skipped,
|
||||||
|
build_severity_zero,
|
||||||
|
)
|
||||||
|
from backend.implementation.postprocessing.calibration import (
|
||||||
|
CalibrationConfig,
|
||||||
|
calibration_config_from_params,
|
||||||
|
interpret_angle_logits,
|
||||||
|
interpret_inflammation_logits,
|
||||||
|
)
|
||||||
|
from backend.implementation.postprocessing.measurement import calculate_thickness
|
||||||
|
from backend.implementation.postprocessing.overlay import COLOR_MAP_POST, COLOR_MAP_SUP, create_overlay
|
||||||
|
from backend.implementation.postprocessing.severity import calculate_severity
|
||||||
|
from backend.implementation.preprocessing.clahe import apply_clahe
|
||||||
|
from backend.services import cv_result_cache
|
||||||
|
from backend.services import triton_runtime_service as triton_runtime
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
SEGMENT_CLASSES_SUP = {
|
||||||
|
0: "background",
|
||||||
|
1: "effusion",
|
||||||
|
2: "fat",
|
||||||
|
3: "fat-pat",
|
||||||
|
4: "femur",
|
||||||
|
5: "synovium",
|
||||||
|
6: "tendon",
|
||||||
|
}
|
||||||
|
SEGMENT_CLASSES_POST = {
|
||||||
|
0: "background",
|
||||||
|
1: "fat",
|
||||||
|
2: "tendon",
|
||||||
|
3: "muscle",
|
||||||
|
4: "femur",
|
||||||
|
5: "artery",
|
||||||
|
6: "baker's cyst",
|
||||||
|
}
|
||||||
|
|
||||||
|
_triton_pipeline_lock = asyncio.Lock()
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class CvInferenceOptions:
|
||||||
|
calibration: CalibrationConfig | None = None
|
||||||
|
model_versions: dict[str, str] | None = None
|
||||||
|
use_cache: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class CvBatchResult:
|
||||||
|
results: list[dict[str, Any]]
|
||||||
|
triton_infer_calls: int
|
||||||
|
triton_infer_modes: list[str]
|
||||||
|
|
||||||
|
|
||||||
|
def _encode_image_to_data_url(image_pil: Image.Image) -> str:
|
||||||
|
buffered = io.BytesIO()
|
||||||
|
image_pil.save(buffered, format="PNG")
|
||||||
|
encoded = base64.b64encode(buffered.getvalue()).decode()
|
||||||
|
return f"data:image/png;base64,{encoded}"
|
||||||
|
|
||||||
|
|
||||||
|
def _build_inflammation_payload(logits_row: np.ndarray, config: CalibrationConfig) -> dict:
|
||||||
|
interpreted = interpret_inflammation_logits(logits_row, config)
|
||||||
|
return {
|
||||||
|
"detected": interpreted["detected"],
|
||||||
|
"confidence": interpreted["confidence"],
|
||||||
|
"calibration": interpreted["calibration"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _logits_to_masks(
|
||||||
|
logits: np.ndarray,
|
||||||
|
angle_class: str,
|
||||||
|
image_size: tuple[int, int],
|
||||||
|
batch_idx: int = 0,
|
||||||
|
) -> dict[str, np.ndarray]:
|
||||||
|
logits_arr = np.asarray(logits, dtype=np.float32)
|
||||||
|
while logits_arr.ndim < 4:
|
||||||
|
logits_arr = np.expand_dims(logits_arr, 0)
|
||||||
|
if logits_arr.ndim != 4:
|
||||||
|
raise ValueError(f"Unexpected segmentation logits shape: {logits_arr.shape}")
|
||||||
|
if batch_idx >= logits_arr.shape[0]:
|
||||||
|
raise IndexError(f"batch_idx {batch_idx} out of range for shape {logits_arr.shape}")
|
||||||
|
|
||||||
|
preds_lowres = logits_arr.argmax(axis=1)[batch_idx]
|
||||||
|
width, height = image_size
|
||||||
|
preds = cv2.resize(
|
||||||
|
preds_lowres.astype(np.uint8),
|
||||||
|
(width, height),
|
||||||
|
interpolation=cv2.INTER_NEAREST,
|
||||||
|
)
|
||||||
|
|
||||||
|
angle_type = get_angle_type(angle_class)
|
||||||
|
class_map = SEGMENT_CLASSES_SUP if angle_type == "sup" else SEGMENT_CLASSES_POST
|
||||||
|
|
||||||
|
masks: dict[str, np.ndarray] = {}
|
||||||
|
for class_id, class_name in class_map.items():
|
||||||
|
masks[class_name] = (preds == class_id).astype(np.uint8)
|
||||||
|
return masks
|
||||||
|
|
||||||
|
|
||||||
|
def _build_color_legend(classes_detected: list[str], angle_type: str) -> dict[str, list[int]]:
|
||||||
|
color_map = COLOR_MAP_SUP if angle_type == "sup" else COLOR_MAP_POST
|
||||||
|
legend: dict[str, list[int]] = {}
|
||||||
|
for class_name in classes_detected:
|
||||||
|
if class_name in color_map:
|
||||||
|
legend[class_name] = color_map[class_name]
|
||||||
|
return legend
|
||||||
|
|
||||||
|
|
||||||
|
def _build_segmentation_result(
|
||||||
|
image_pil: Image.Image,
|
||||||
|
logits: np.ndarray,
|
||||||
|
angle_class: str,
|
||||||
|
seg_model: str,
|
||||||
|
*,
|
||||||
|
frame_id: str | None,
|
||||||
|
inflammation: dict,
|
||||||
|
angle_payload: dict,
|
||||||
|
enhanced_data_url: str,
|
||||||
|
inflam_model: str,
|
||||||
|
angle_model: str,
|
||||||
|
) -> dict:
|
||||||
|
angle_type = get_angle_type(angle_class)
|
||||||
|
masks = _logits_to_masks(logits, angle_class, image_pil.size)
|
||||||
|
measurement = calculate_thickness(masks, image_pil.size)
|
||||||
|
severity = calculate_severity(masks, image_pil.size)
|
||||||
|
overlay = create_overlay(image_pil, masks, measurement, angle_type)
|
||||||
|
|
||||||
|
classes_detected = [name for name, mask in masks.items() if np.sum(mask) > 0 and name != "background"]
|
||||||
|
color_legend = _build_color_legend(classes_detected, angle_type)
|
||||||
|
|
||||||
|
result: dict[str, Any] = {
|
||||||
|
"success": True,
|
||||||
|
"angle": angle_payload,
|
||||||
|
"inflammation": inflammation,
|
||||||
|
"measurement": measurement,
|
||||||
|
"severity": severity,
|
||||||
|
"segmentation": {
|
||||||
|
"performed": True,
|
||||||
|
"angle_type": angle_type,
|
||||||
|
"classes_detected": classes_detected,
|
||||||
|
"color_legend": color_legend,
|
||||||
|
},
|
||||||
|
"images": {
|
||||||
|
"enhanced": enhanced_data_url,
|
||||||
|
"segmented": _encode_image_to_data_url(overlay),
|
||||||
|
},
|
||||||
|
"models_used": {
|
||||||
|
"angle": angle_model,
|
||||||
|
"inflammation": inflam_model,
|
||||||
|
"segmentation": seg_model,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if frame_id is not None:
|
||||||
|
result["frame_id"] = frame_id
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
async def _run_spec_cv_pipeline_single(
|
||||||
|
image_pil: Image.Image,
|
||||||
|
*,
|
||||||
|
frame_id: str | None,
|
||||||
|
options: CvInferenceOptions,
|
||||||
|
) -> tuple[dict[str, Any], str, int]:
|
||||||
|
"""
|
||||||
|
Per-image spec path:
|
||||||
|
CLAHE → angle → (post-trans|sup-up-long only) inflammation → conditional segmentation.
|
||||||
|
"""
|
||||||
|
config = options.calibration or CalibrationConfig()
|
||||||
|
model_versions = options.model_versions
|
||||||
|
angle_model = get_model_name("angle", model_versions)
|
||||||
|
inflam_model = get_model_name("inflammation", model_versions)
|
||||||
|
|
||||||
|
triton_calls = 0
|
||||||
|
modes: list[str] = []
|
||||||
|
|
||||||
|
enhanced_pil = apply_clahe(image_pil)
|
||||||
|
enhanced_data_url = _encode_image_to_data_url(enhanced_pil)
|
||||||
|
|
||||||
|
angle_logits, angle_mode, angle_calls = await triton_runtime.infer_angle_logits_single(
|
||||||
|
image_pil, angle_model
|
||||||
|
)
|
||||||
|
modes.append(angle_mode)
|
||||||
|
triton_calls += angle_calls
|
||||||
|
|
||||||
|
angle_interpreted = interpret_angle_logits(angle_logits, config)
|
||||||
|
angle_payload = {
|
||||||
|
"class": angle_interpreted["class"],
|
||||||
|
"confidence": angle_interpreted["confidence"],
|
||||||
|
"calibration": angle_interpreted["calibration"],
|
||||||
|
}
|
||||||
|
|
||||||
|
result: dict[str, Any] = {
|
||||||
|
"success": True,
|
||||||
|
"angle": angle_payload,
|
||||||
|
"models_used": {"angle": angle_model},
|
||||||
|
"images": {"enhanced": enhanced_data_url},
|
||||||
|
}
|
||||||
|
if frame_id is not None:
|
||||||
|
result["frame_id"] = frame_id
|
||||||
|
|
||||||
|
angle_class = angle_interpreted["class"]
|
||||||
|
if angle_class not in BRANCH_ANGLE_CLASSES:
|
||||||
|
result["segmentation"] = build_segmentation_skipped("angle_only")
|
||||||
|
result["severity"] = build_severity_zero("angle_only")
|
||||||
|
return result, "+".join(modes), triton_calls
|
||||||
|
|
||||||
|
inflam_logits, inflam_mode, inflam_calls = await triton_runtime.infer_inflammation_logits_single(
|
||||||
|
image_pil, inflam_model
|
||||||
|
)
|
||||||
|
modes.append(inflam_mode)
|
||||||
|
triton_calls += inflam_calls
|
||||||
|
|
||||||
|
inflammation = _build_inflammation_payload(inflam_logits, config)
|
||||||
|
result["inflammation"] = inflammation
|
||||||
|
result["models_used"]["inflammation"] = inflam_model
|
||||||
|
|
||||||
|
if not inflammation.get("detected"):
|
||||||
|
result["segmentation"] = build_segmentation_skipped("no_inflammation")
|
||||||
|
result["severity"] = build_severity_zero("no_inflammation")
|
||||||
|
return result, "+".join(modes), triton_calls
|
||||||
|
|
||||||
|
seg_model = get_segmentation_model(angle_class, model_versions)
|
||||||
|
seg_logits, seg_mode, seg_calls = await triton_runtime.infer_segmentation_logits_single(
|
||||||
|
image_pil, seg_model
|
||||||
|
)
|
||||||
|
modes.append(seg_mode)
|
||||||
|
triton_calls += seg_calls
|
||||||
|
|
||||||
|
seg_result = _build_segmentation_result(
|
||||||
|
image_pil,
|
||||||
|
seg_logits,
|
||||||
|
angle_class,
|
||||||
|
seg_model,
|
||||||
|
frame_id=frame_id,
|
||||||
|
inflammation=inflammation,
|
||||||
|
angle_payload=angle_payload,
|
||||||
|
enhanced_data_url=enhanced_data_url,
|
||||||
|
inflam_model=inflam_model,
|
||||||
|
angle_model=angle_model,
|
||||||
|
)
|
||||||
|
return seg_result, "+".join(modes), triton_calls
|
||||||
|
|
||||||
|
|
||||||
|
async def run_single(
|
||||||
|
image: Image.Image,
|
||||||
|
*,
|
||||||
|
frame_id: str | None = None,
|
||||||
|
options: CvInferenceOptions | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
opts = options or CvInferenceOptions()
|
||||||
|
result, _, _ = await _run_spec_cv_pipeline_single(image, frame_id=frame_id, options=opts)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
async def _run_batch_uncached(
|
||||||
|
images: list[Image.Image],
|
||||||
|
frame_ids: list[str],
|
||||||
|
options: CvInferenceOptions,
|
||||||
|
) -> CvBatchResult:
|
||||||
|
if not images:
|
||||||
|
return CvBatchResult(results=[], triton_infer_calls=0, triton_infer_modes=[])
|
||||||
|
if len(frame_ids) != len(images):
|
||||||
|
raise ValueError("frame_ids length must match images length")
|
||||||
|
|
||||||
|
async with _triton_pipeline_lock:
|
||||||
|
results: list[dict[str, Any]] = []
|
||||||
|
infer_modes: list[str] = []
|
||||||
|
triton_call_count = 0
|
||||||
|
for image_pil, fid in zip(images, frame_ids, strict=True):
|
||||||
|
item, mode, calls = await _run_spec_cv_pipeline_single(
|
||||||
|
image_pil,
|
||||||
|
frame_id=fid,
|
||||||
|
options=options,
|
||||||
|
)
|
||||||
|
results.append(item)
|
||||||
|
infer_modes.append(mode)
|
||||||
|
triton_call_count += calls
|
||||||
|
return CvBatchResult(
|
||||||
|
results=results,
|
||||||
|
triton_infer_calls=triton_call_count,
|
||||||
|
triton_infer_modes=infer_modes,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def run_batch(
|
||||||
|
images: list[Image.Image],
|
||||||
|
frame_ids: list[str],
|
||||||
|
options: CvInferenceOptions | None = None,
|
||||||
|
) -> CvBatchResult:
|
||||||
|
opts = options or CvInferenceOptions()
|
||||||
|
if not opts.use_cache or not images:
|
||||||
|
return await _run_batch_uncached(images, frame_ids, opts)
|
||||||
|
|
||||||
|
image_hashes = []
|
||||||
|
for image in images:
|
||||||
|
buf = io.BytesIO()
|
||||||
|
image.save(buf, format="PNG")
|
||||||
|
image_hashes.append(cv_result_cache.hash_image_bytes(buf.getvalue()))
|
||||||
|
|
||||||
|
cache_key = cv_result_cache.analyze_batch_cache_key(frame_ids, image_hashes)
|
||||||
|
|
||||||
|
async def compute():
|
||||||
|
return await _run_batch_uncached(images, frame_ids, opts)
|
||||||
|
|
||||||
|
return await cv_result_cache.with_result_cache(cache_key, compute, enabled=opts.use_cache)
|
||||||
|
|
||||||
|
|
||||||
|
def options_from_params(params: dict[str, Any] | None) -> CvInferenceOptions:
|
||||||
|
params = params or {}
|
||||||
|
calibration = calibration_config_from_params(params)
|
||||||
|
model_versions = params.get("model_versions")
|
||||||
|
use_cache = params.get("use_cache", True)
|
||||||
|
return CvInferenceOptions(
|
||||||
|
calibration=calibration,
|
||||||
|
model_versions=model_versions,
|
||||||
|
use_cache=use_cache,
|
||||||
|
)
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
"""In-memory CV inference result cache with in-flight request coalescing."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import hashlib
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import time
|
||||||
|
from typing import Any, Awaitable, Callable, TypeVar
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
CV_PIPELINE_VERSION = os.getenv("CV_PIPELINE_VERSION", "poc-v2-spec-cv-seg-norm")
|
||||||
|
CV_RESULT_CACHE_TTL_S = float(os.getenv("CV_RESULT_CACHE_TTL_S", "3600"))
|
||||||
|
CV_CACHE_ENABLED = os.getenv("CV_CACHE_ENABLED", "true").lower() in {"1", "true", "yes"}
|
||||||
|
|
||||||
|
T = TypeVar("T")
|
||||||
|
|
||||||
|
_result_cache: dict[str, tuple[float, Any]] = {}
|
||||||
|
_inflight: dict[str, asyncio.Future] = {}
|
||||||
|
|
||||||
|
|
||||||
|
def hash_image_bytes(raw: bytes) -> str:
|
||||||
|
return hashlib.sha256(raw).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def analyze_batch_cache_key(frame_ids: list[str], image_hashes: list[str]) -> str:
|
||||||
|
pairs = sorted(zip(frame_ids, image_hashes, strict=True), key=lambda item: item[0])
|
||||||
|
payload = "|".join(f"{frame_id}:{digest}" for frame_id, digest in pairs)
|
||||||
|
return f"analyze|{CV_PIPELINE_VERSION}|{payload}"
|
||||||
|
|
||||||
|
|
||||||
|
async def with_result_cache(cache_key: str, compute: Callable[[], Awaitable[T]], *, enabled: bool = True) -> T:
|
||||||
|
if not enabled or not CV_CACHE_ENABLED:
|
||||||
|
return await compute()
|
||||||
|
|
||||||
|
now = time.monotonic()
|
||||||
|
cached = _result_cache.get(cache_key)
|
||||||
|
if cached and cached[0] > now:
|
||||||
|
logger.info("CV cache HIT: %s", cache_key[:96])
|
||||||
|
return cached[1]
|
||||||
|
|
||||||
|
inflight = _inflight.get(cache_key)
|
||||||
|
if inflight is not None:
|
||||||
|
logger.info("CV in-flight coalesce: %s", cache_key[:96])
|
||||||
|
return await inflight
|
||||||
|
|
||||||
|
loop = asyncio.get_running_loop()
|
||||||
|
fut: asyncio.Future = loop.create_future()
|
||||||
|
_inflight[cache_key] = fut
|
||||||
|
try:
|
||||||
|
result = await compute()
|
||||||
|
_result_cache[cache_key] = (time.monotonic() + CV_RESULT_CACHE_TTL_S, result)
|
||||||
|
fut.set_result(result)
|
||||||
|
logger.info("CV cache STORE: %s", cache_key[:96])
|
||||||
|
return result
|
||||||
|
except Exception as exc:
|
||||||
|
if not fut.done():
|
||||||
|
fut.set_exception(exc)
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
_inflight.pop(cache_key, None)
|
||||||
|
|
||||||
|
|
||||||
|
def cache_stats() -> dict[str, int | bool | float | str]:
|
||||||
|
return {
|
||||||
|
"cache_enabled": CV_CACHE_ENABLED,
|
||||||
|
"pipeline_version": CV_PIPELINE_VERSION,
|
||||||
|
"cache_ttl_s": CV_RESULT_CACHE_TTL_S,
|
||||||
|
"cache_entries": len(_result_cache),
|
||||||
|
"inflight_batches": len(_inflight),
|
||||||
|
}
|
||||||
@@ -0,0 +1,88 @@
|
|||||||
|
"""EmbeddingGemma-compatible embed endpoint for episodic memory and RAG queries."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import logging
|
||||||
|
import math
|
||||||
|
import os
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
EMBEDDING_DIMENSIONS = 768
|
||||||
|
|
||||||
|
|
||||||
|
def _format_embed_input(text: str, task: str, title: str | None = None) -> str:
|
||||||
|
if task == "retrieval-document":
|
||||||
|
title_value = title.strip() if title and title.strip() else "none"
|
||||||
|
return f"title: {title_value} | text: {text}"
|
||||||
|
return f"task: search result | query: {text}"
|
||||||
|
|
||||||
|
|
||||||
|
def deterministic_embed(text: str, dimensions: int = EMBEDDING_DIMENSIONS) -> list[float]:
|
||||||
|
"""PoC fallback — matches gemma4_e2b deterministicEmbed (SHA-256 + char histogram)."""
|
||||||
|
vec = [0.0] * dimensions
|
||||||
|
normalized = text.lower().strip()
|
||||||
|
if not normalized:
|
||||||
|
return vec
|
||||||
|
|
||||||
|
for index, char in enumerate(normalized):
|
||||||
|
code = ord(char)
|
||||||
|
bucket = (code * (index + 17)) % dimensions
|
||||||
|
vec[bucket] += 1.0
|
||||||
|
|
||||||
|
digest = hashlib.sha256(normalized.encode("utf-8")).digest()
|
||||||
|
for i in range(dimensions):
|
||||||
|
vec[i] += digest[i % len(digest)] / 255.0
|
||||||
|
|
||||||
|
norm = math.sqrt(sum(value * value for value in vec))
|
||||||
|
if norm == 0:
|
||||||
|
return vec
|
||||||
|
return [value / norm for value in vec]
|
||||||
|
|
||||||
|
|
||||||
|
def _try_gemma_embedder(formatted: str) -> list[float] | None:
|
||||||
|
"""Optional real EmbeddingGemma via knowledge ingestion pipeline."""
|
||||||
|
if os.getenv("EMBED_QUERY_MOCK") == "1":
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
from knowledge.implementation.ingestion.embedding import GemmaEmbedder, EmbedTask
|
||||||
|
|
||||||
|
embedder = GemmaEmbedder()
|
||||||
|
task = (
|
||||||
|
EmbedTask.RETRIEVAL_DOCUMENT
|
||||||
|
if formatted.startswith("title:")
|
||||||
|
else EmbedTask.RETRIEVAL_QUERY
|
||||||
|
)
|
||||||
|
raw = formatted
|
||||||
|
title = None
|
||||||
|
if task == EmbedTask.RETRIEVAL_DOCUMENT and "| text: " in formatted:
|
||||||
|
prefix, body = formatted.split("| text: ", 1)
|
||||||
|
title = prefix.replace("title:", "").strip()
|
||||||
|
raw = body
|
||||||
|
vector = embedder.embed(raw, task, title=title if title and title != "none" else None)
|
||||||
|
return vector.tolist()
|
||||||
|
except Exception as exc:
|
||||||
|
logger.debug("GemmaEmbedder unavailable: %s", exc)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def embed_text(
|
||||||
|
text: str,
|
||||||
|
task: Literal["retrieval-query", "retrieval-document"] = "retrieval-query",
|
||||||
|
title: str | None = None,
|
||||||
|
) -> dict[str, object]:
|
||||||
|
formatted = _format_embed_input(text, task, title)
|
||||||
|
vector = _try_gemma_embedder(formatted)
|
||||||
|
if vector is not None:
|
||||||
|
return {"vector": vector, "model": "embeddinggemma-300m", "source": "gemma"}
|
||||||
|
|
||||||
|
if os.getenv("EMBED_QUERY_MOCK") == "1":
|
||||||
|
logger.warning("Using deterministic embed fallback (EMBED_QUERY_MOCK or no embedder)")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"vector": deterministic_embed(formatted),
|
||||||
|
"model": "embeddinggemma-300m-deterministic",
|
||||||
|
"source": "deterministic",
|
||||||
|
}
|
||||||
@@ -0,0 +1,384 @@
|
|||||||
|
"""Triton inference runtime — lock, retry, batching with batched→sequential fallback."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import time
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import requests
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
from backend.implementation.adapters.triton_adapter import TritonAdapter
|
||||||
|
from backend.implementation.preprocessing.tensor_prep import (
|
||||||
|
prepare_angle_tensor,
|
||||||
|
prepare_inflammation_tensor,
|
||||||
|
prepare_segmentation_tensor,
|
||||||
|
)
|
||||||
|
from backend.implementation.triton_batch import TRITON_MAX_BATCH_SIZE, chunk_sequence
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
INPUT_NAME = "input_image"
|
||||||
|
OUTPUT_NAME = "logits"
|
||||||
|
|
||||||
|
TRITON_INFER_TIMEOUT = float(os.getenv("TRITON_INFER_TIMEOUT", "120"))
|
||||||
|
TRITON_INFER_RETRIES = int(os.getenv("TRITON_INFER_RETRIES", "5"))
|
||||||
|
TRITON_RETRY_BASE_S = float(os.getenv("TRITON_RETRY_BASE_S", "4"))
|
||||||
|
TRITON_USE_BATCH_INFER = os.getenv("TRITON_USE_BATCH_INFER", "true").lower()
|
||||||
|
RETRYABLE_STATUS = {429, 502, 503, 504}
|
||||||
|
|
||||||
|
_triton_infer_lock = asyncio.Lock()
|
||||||
|
_adapter: TritonAdapter | None = None
|
||||||
|
_adapter_endpoint: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def get_triton_endpoint() -> str:
|
||||||
|
return os.getenv("TRITON_ENDPOINT", "http://localhost:8000").rstrip("/")
|
||||||
|
|
||||||
|
|
||||||
|
def _get_adapter() -> TritonAdapter:
|
||||||
|
global _adapter, _adapter_endpoint
|
||||||
|
endpoint = get_triton_endpoint()
|
||||||
|
if _adapter is None or _adapter_endpoint != endpoint:
|
||||||
|
_adapter = TritonAdapter(endpoint_url=endpoint, timeout=TRITON_INFER_TIMEOUT)
|
||||||
|
_adapter_endpoint = endpoint
|
||||||
|
return _adapter
|
||||||
|
|
||||||
|
|
||||||
|
def _retry_backoff_seconds(attempt: int) -> float:
|
||||||
|
return TRITON_RETRY_BASE_S * (2 ** (attempt - 1))
|
||||||
|
|
||||||
|
|
||||||
|
def _should_try_batched_infer(image_count: int) -> bool:
|
||||||
|
if image_count <= 1:
|
||||||
|
return True
|
||||||
|
if TRITON_USE_BATCH_INFER in {"1", "true", "yes"}:
|
||||||
|
return True
|
||||||
|
if TRITON_USE_BATCH_INFER in {"0", "false", "no"}:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def _logits_array_from_adapter(result: dict) -> np.ndarray:
|
||||||
|
logits = result.get(OUTPUT_NAME, [])
|
||||||
|
if not logits:
|
||||||
|
raise ValueError(f"Empty {OUTPUT_NAME} in Triton response")
|
||||||
|
return np.asarray(logits, dtype=np.float32)
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_sync_with_retry(
|
||||||
|
model_name: str,
|
||||||
|
batch_tensor: np.ndarray,
|
||||||
|
*,
|
||||||
|
operation: str,
|
||||||
|
max_retries: int | None = None,
|
||||||
|
) -> np.ndarray:
|
||||||
|
adapter = _get_adapter()
|
||||||
|
attempts = max_retries if max_retries is not None else TRITON_INFER_RETRIES
|
||||||
|
last_error: Exception | None = None
|
||||||
|
|
||||||
|
inputs = {
|
||||||
|
INPUT_NAME: {
|
||||||
|
"data": batch_tensor,
|
||||||
|
"shape": list(batch_tensor.shape),
|
||||||
|
"datatype": "FP32",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for attempt in range(1, attempts + 1):
|
||||||
|
try:
|
||||||
|
result = adapter._infer_sync(model_name, inputs, outputs=[OUTPUT_NAME])
|
||||||
|
if attempt > 1:
|
||||||
|
logger.info("%s succeeded on attempt %s/%s", operation, attempt, attempts)
|
||||||
|
return _logits_array_from_adapter(result)
|
||||||
|
except requests.HTTPError as exc:
|
||||||
|
status = exc.response.status_code if exc.response is not None else None
|
||||||
|
if status not in RETRYABLE_STATUS:
|
||||||
|
raise
|
||||||
|
last_error = exc
|
||||||
|
except (requests.ConnectionError, requests.Timeout) as exc:
|
||||||
|
last_error = exc
|
||||||
|
|
||||||
|
if attempt >= attempts:
|
||||||
|
logger.error("%s failed on final attempt %s/%s: %s", operation, attempt, attempts, last_error)
|
||||||
|
break
|
||||||
|
wait_s = _retry_backoff_seconds(attempt)
|
||||||
|
logger.warning(
|
||||||
|
"%s attempt %s/%s failed (%s); exponential retry in %.1fs",
|
||||||
|
operation,
|
||||||
|
attempt,
|
||||||
|
attempts,
|
||||||
|
last_error,
|
||||||
|
wait_s,
|
||||||
|
)
|
||||||
|
time.sleep(wait_s)
|
||||||
|
|
||||||
|
assert last_error is not None
|
||||||
|
raise last_error
|
||||||
|
|
||||||
|
|
||||||
|
def _stack_angle_tensors(images: list[Image.Image]) -> np.ndarray:
|
||||||
|
if not images:
|
||||||
|
raise ValueError("images must not be empty")
|
||||||
|
if len(images) > TRITON_MAX_BATCH_SIZE:
|
||||||
|
raise ValueError(f"At most {TRITON_MAX_BATCH_SIZE} images per Triton batch, got {len(images)}")
|
||||||
|
tensors = [prepare_angle_tensor(img) for img in images]
|
||||||
|
return np.concatenate(tensors, axis=0).astype(np.float32)
|
||||||
|
|
||||||
|
|
||||||
|
def _stack_inflammation_tensors(images: list[Image.Image]) -> np.ndarray:
|
||||||
|
if not images:
|
||||||
|
raise ValueError("images must not be empty")
|
||||||
|
if len(images) > TRITON_MAX_BATCH_SIZE:
|
||||||
|
raise ValueError(f"At most {TRITON_MAX_BATCH_SIZE} images per Triton batch, got {len(images)}")
|
||||||
|
tensors = [prepare_inflammation_tensor(img) for img in images]
|
||||||
|
return np.concatenate(tensors, axis=0).astype(np.float32)
|
||||||
|
|
||||||
|
|
||||||
|
def _stack_segmentation_tensors(images: list[Image.Image]) -> np.ndarray:
|
||||||
|
if not images:
|
||||||
|
raise ValueError("images must not be empty")
|
||||||
|
if len(images) > TRITON_MAX_BATCH_SIZE:
|
||||||
|
raise ValueError(f"At most {TRITON_MAX_BATCH_SIZE} images per Triton batch, got {len(images)}")
|
||||||
|
tensors = [prepare_segmentation_tensor(img) for img in images]
|
||||||
|
return np.concatenate(tensors, axis=0).astype(np.float32)
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_batched_angle_logits(logits: np.ndarray, expected_count: int) -> np.ndarray:
|
||||||
|
if logits.ndim == 1:
|
||||||
|
logits = np.expand_dims(logits, axis=0)
|
||||||
|
if logits.ndim != 2:
|
||||||
|
raise ValueError(f"Unexpected batched angle logits shape: {logits.shape}")
|
||||||
|
if logits.shape[0] != expected_count:
|
||||||
|
raise ValueError(
|
||||||
|
f"Triton returned batch {logits.shape[0]} but expected {expected_count} angle rows",
|
||||||
|
)
|
||||||
|
return logits
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_batched_segmentation_logits(logits: np.ndarray, expected_count: int) -> np.ndarray:
|
||||||
|
if logits.ndim == 3:
|
||||||
|
logits = np.expand_dims(logits, axis=0)
|
||||||
|
if logits.ndim != 4:
|
||||||
|
raise ValueError(f"Unexpected batched segmentation logits shape: {logits.shape}")
|
||||||
|
if logits.shape[0] != expected_count:
|
||||||
|
raise ValueError(
|
||||||
|
f"Triton returned batch {logits.shape[0]} but expected {expected_count} images",
|
||||||
|
)
|
||||||
|
return logits
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_angle_logits_batch(
|
||||||
|
images: list[Image.Image],
|
||||||
|
model_name: str,
|
||||||
|
*,
|
||||||
|
max_retries: int | None = None,
|
||||||
|
) -> np.ndarray:
|
||||||
|
batch_tensor = _stack_angle_tensors(images)
|
||||||
|
logits = _infer_sync_with_retry(
|
||||||
|
model_name,
|
||||||
|
batch_tensor,
|
||||||
|
operation=f"Triton angle batch×{len(images)} ({model_name})",
|
||||||
|
max_retries=max_retries,
|
||||||
|
)
|
||||||
|
return _normalize_batched_angle_logits(logits, len(images))
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_angle_logits_sequential(images: list[Image.Image], model_name: str) -> np.ndarray:
|
||||||
|
rows: list[np.ndarray] = []
|
||||||
|
for index, image in enumerate(images, start=1):
|
||||||
|
logits = _infer_angle_logits_batch([image], model_name)
|
||||||
|
row = logits[0] if logits.ndim == 2 else logits
|
||||||
|
rows.append(row)
|
||||||
|
logger.info("Triton sequential angle infer %s/%s complete for %s", index, len(images), model_name)
|
||||||
|
return np.stack(rows, axis=0)
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_angle_logits_chunk(images: list[Image.Image], model_name: str) -> tuple[np.ndarray, Literal["batched", "sequential"]]:
|
||||||
|
if not _should_try_batched_infer(len(images)):
|
||||||
|
return _infer_angle_logits_sequential(images, model_name), "sequential"
|
||||||
|
try:
|
||||||
|
return _infer_angle_logits_batch(images, model_name, max_retries=1), "batched"
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(
|
||||||
|
"Batched angle infer×%s failed (%s); falling back to sequential single-image calls",
|
||||||
|
len(images),
|
||||||
|
exc,
|
||||||
|
)
|
||||||
|
return _infer_angle_logits_sequential(images, model_name), "sequential"
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_inflammation_logits_batch(
|
||||||
|
images: list[Image.Image],
|
||||||
|
model_name: str,
|
||||||
|
*,
|
||||||
|
max_retries: int | None = None,
|
||||||
|
) -> np.ndarray:
|
||||||
|
batch_tensor = _stack_inflammation_tensors(images)
|
||||||
|
logits = _infer_sync_with_retry(
|
||||||
|
model_name,
|
||||||
|
batch_tensor,
|
||||||
|
operation=f"Triton inflammation batch×{len(images)} ({model_name})",
|
||||||
|
max_retries=max_retries,
|
||||||
|
)
|
||||||
|
return _normalize_batched_angle_logits(logits, len(images))
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_inflammation_logits_sequential(images: list[Image.Image], model_name: str) -> np.ndarray:
|
||||||
|
rows: list[np.ndarray] = []
|
||||||
|
for index, image in enumerate(images, start=1):
|
||||||
|
logits = _infer_inflammation_logits_batch([image], model_name)
|
||||||
|
row = logits[0] if logits.ndim == 2 else logits
|
||||||
|
rows.append(row)
|
||||||
|
logger.info(
|
||||||
|
"Triton sequential inflammation infer %s/%s complete for %s",
|
||||||
|
index,
|
||||||
|
len(images),
|
||||||
|
model_name,
|
||||||
|
)
|
||||||
|
return np.stack(rows, axis=0)
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_inflammation_logits_chunk(
|
||||||
|
images: list[Image.Image],
|
||||||
|
model_name: str,
|
||||||
|
) -> tuple[np.ndarray, Literal["batched", "sequential"]]:
|
||||||
|
if not _should_try_batched_infer(len(images)):
|
||||||
|
return _infer_inflammation_logits_sequential(images, model_name), "sequential"
|
||||||
|
try:
|
||||||
|
return _infer_inflammation_logits_batch(images, model_name, max_retries=1), "batched"
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(
|
||||||
|
"Batched inflammation infer×%s failed (%s); falling back to sequential",
|
||||||
|
len(images),
|
||||||
|
exc,
|
||||||
|
)
|
||||||
|
return _infer_inflammation_logits_sequential(images, model_name), "sequential"
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_segmentation_logits_batch(
|
||||||
|
images: list[Image.Image],
|
||||||
|
model_name: str,
|
||||||
|
*,
|
||||||
|
max_retries: int | None = None,
|
||||||
|
) -> np.ndarray:
|
||||||
|
batch_tensor = _stack_segmentation_tensors(images)
|
||||||
|
logits = _infer_sync_with_retry(
|
||||||
|
model_name,
|
||||||
|
batch_tensor,
|
||||||
|
operation=f"Triton segmentation batch×{len(images)} ({model_name})",
|
||||||
|
max_retries=max_retries,
|
||||||
|
)
|
||||||
|
return _normalize_batched_segmentation_logits(logits, len(images))
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_segmentation_logits_sequential(images: list[Image.Image], model_name: str) -> np.ndarray:
|
||||||
|
rows: list[np.ndarray] = []
|
||||||
|
for index, image in enumerate(images, start=1):
|
||||||
|
logits = _infer_segmentation_logits_batch([image], model_name)
|
||||||
|
row = logits[0] if logits.ndim == 4 else logits
|
||||||
|
rows.append(row)
|
||||||
|
logger.info(
|
||||||
|
"Triton sequential segmentation infer %s/%s complete for %s",
|
||||||
|
index,
|
||||||
|
len(images),
|
||||||
|
model_name,
|
||||||
|
)
|
||||||
|
return np.stack(rows, axis=0)
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_segmentation_logits_chunk(
|
||||||
|
images: list[Image.Image],
|
||||||
|
model_name: str,
|
||||||
|
) -> tuple[np.ndarray, Literal["batched", "sequential"]]:
|
||||||
|
if not _should_try_batched_infer(len(images)):
|
||||||
|
return _infer_segmentation_logits_sequential(images, model_name), "sequential"
|
||||||
|
try:
|
||||||
|
return _infer_segmentation_logits_batch(images, model_name, max_retries=1), "batched"
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(
|
||||||
|
"Batched segmentation infer×%s failed (%s); falling back to sequential",
|
||||||
|
len(images),
|
||||||
|
exc,
|
||||||
|
)
|
||||||
|
return _infer_segmentation_logits_sequential(images, model_name), "sequential"
|
||||||
|
|
||||||
|
|
||||||
|
async def infer_angle_logits(
|
||||||
|
images: list[Image.Image],
|
||||||
|
model_name: str,
|
||||||
|
) -> tuple[np.ndarray, Literal["batched", "sequential"], int]:
|
||||||
|
"""Run angle classification under the global Triton lock. Returns (logits, mode, call_count)."""
|
||||||
|
async with _triton_infer_lock:
|
||||||
|
all_logits: list[np.ndarray] = []
|
||||||
|
modes: list[str] = []
|
||||||
|
call_count = 0
|
||||||
|
for chunk in chunk_sequence(images):
|
||||||
|
logits, mode = await asyncio.to_thread(_infer_angle_logits_chunk, chunk, model_name)
|
||||||
|
all_logits.append(logits)
|
||||||
|
modes.append(mode)
|
||||||
|
call_count += 1 if mode == "batched" else len(chunk)
|
||||||
|
combined = np.concatenate(all_logits, axis=0) if len(all_logits) > 1 else all_logits[0]
|
||||||
|
infer_mode: Literal["batched", "sequential"] = (
|
||||||
|
"sequential" if any(m == "sequential" for m in modes) else "batched"
|
||||||
|
)
|
||||||
|
return combined, infer_mode, call_count
|
||||||
|
|
||||||
|
|
||||||
|
async def infer_inflammation_logits(
|
||||||
|
images: list[Image.Image],
|
||||||
|
model_name: str,
|
||||||
|
) -> tuple[np.ndarray, Literal["batched", "sequential"], int]:
|
||||||
|
async with _triton_infer_lock:
|
||||||
|
all_logits: list[np.ndarray] = []
|
||||||
|
modes: list[str] = []
|
||||||
|
call_count = 0
|
||||||
|
for chunk in chunk_sequence(images):
|
||||||
|
logits, mode = await asyncio.to_thread(_infer_inflammation_logits_chunk, chunk, model_name)
|
||||||
|
all_logits.append(logits)
|
||||||
|
modes.append(mode)
|
||||||
|
call_count += 1 if mode == "batched" else len(chunk)
|
||||||
|
combined = np.concatenate(all_logits, axis=0) if len(all_logits) > 1 else all_logits[0]
|
||||||
|
infer_mode: Literal["batched", "sequential"] = (
|
||||||
|
"sequential" if any(m == "sequential" for m in modes) else "batched"
|
||||||
|
)
|
||||||
|
return combined, infer_mode, call_count
|
||||||
|
|
||||||
|
|
||||||
|
async def infer_segmentation_logits(
|
||||||
|
images: list[Image.Image],
|
||||||
|
model_name: str,
|
||||||
|
) -> tuple[np.ndarray, Literal["batched", "sequential"], int]:
|
||||||
|
async with _triton_infer_lock:
|
||||||
|
all_logits: list[np.ndarray] = []
|
||||||
|
modes: list[str] = []
|
||||||
|
call_count = 0
|
||||||
|
for chunk in chunk_sequence(images):
|
||||||
|
logits, mode = await asyncio.to_thread(_infer_segmentation_logits_chunk, chunk, model_name)
|
||||||
|
all_logits.append(logits)
|
||||||
|
modes.append(mode)
|
||||||
|
call_count += 1 if mode == "batched" else len(chunk)
|
||||||
|
combined = np.concatenate(all_logits, axis=0) if len(all_logits) > 1 else all_logits[0]
|
||||||
|
infer_mode: Literal["batched", "sequential"] = (
|
||||||
|
"sequential" if any(m == "sequential" for m in modes) else "batched"
|
||||||
|
)
|
||||||
|
return combined, infer_mode, call_count
|
||||||
|
|
||||||
|
|
||||||
|
async def infer_angle_logits_single(image: Image.Image, model_name: str) -> tuple[np.ndarray, str, int]:
|
||||||
|
logits, mode, calls = await infer_angle_logits([image], model_name)
|
||||||
|
return logits[0], f"angle:{mode}", calls
|
||||||
|
|
||||||
|
|
||||||
|
async def infer_inflammation_logits_single(image: Image.Image, model_name: str) -> tuple[np.ndarray, str, int]:
|
||||||
|
logits, mode, calls = await infer_inflammation_logits([image], model_name)
|
||||||
|
return logits[0], f"inflam:{mode}", calls
|
||||||
|
|
||||||
|
|
||||||
|
async def infer_segmentation_logits_single(image: Image.Image, model_name: str) -> tuple[np.ndarray, str, int]:
|
||||||
|
logits, mode, calls = await infer_segmentation_logits([image], model_name)
|
||||||
|
row = logits[0] if logits.ndim == 4 else logits
|
||||||
|
return row, f"seg:{mode}", calls
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
"""Warm up Modal Triton on CV service startup — reduces cold-start 502s and first-frame latency."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
from backend.implementation.config import get_model_name, get_segmentation_model
|
||||||
|
from backend.services import triton_runtime_service as triton_runtime
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _warmup_model_versions() -> dict[str, str]:
|
||||||
|
versions: dict[str, str] = {}
|
||||||
|
if angle := os.getenv("ANGLE_MODEL"):
|
||||||
|
versions["angle"] = angle
|
||||||
|
else:
|
||||||
|
versions["angle"] = "angle_classify_resnet50"
|
||||||
|
if inflam := os.getenv("INFLAMMATION_MODEL"):
|
||||||
|
versions["inflammation"] = inflam
|
||||||
|
return versions
|
||||||
|
|
||||||
|
|
||||||
|
async def warmup_triton_models() -> None:
|
||||||
|
if os.getenv("CV_SKIP_TRITON_WARMUP", "").lower() in {"1", "true", "yes"}:
|
||||||
|
logger.info("Triton warmup skipped (CV_SKIP_TRITON_WARMUP)")
|
||||||
|
return
|
||||||
|
|
||||||
|
model_versions = _warmup_model_versions()
|
||||||
|
angle_model = get_model_name("angle", model_versions)
|
||||||
|
inflam_model = get_model_name("inflammation", model_versions)
|
||||||
|
seg_model = get_segmentation_model("sup-up-long", model_versions)
|
||||||
|
|
||||||
|
img224 = Image.new("RGB", (224, 224), color=(128, 128, 128))
|
||||||
|
img512 = Image.new("RGB", (512, 512), color=(128, 128, 128))
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
"Warming up Triton (angle=%s, inflammation=%s, segmentation=%s)…",
|
||||||
|
angle_model,
|
||||||
|
inflam_model,
|
||||||
|
seg_model,
|
||||||
|
)
|
||||||
|
await triton_runtime.infer_angle_logits_single(img224, angle_model)
|
||||||
|
await triton_runtime.infer_inflammation_logits_single(img224, inflam_model)
|
||||||
|
await triton_runtime.infer_segmentation_logits_single(img512, seg_model)
|
||||||
|
logger.info("Triton warmup complete")
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
# Agent Tools BFF Contract
|
||||||
|
|
||||||
|
Base path: `/api/v1/agent/tools`
|
||||||
|
|
||||||
|
## POST /exa/search
|
||||||
|
|
||||||
|
Proxies Exa `/search` with server-held `EXA_API_KEY`.
|
||||||
|
|
||||||
|
**Request**
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"query": "synovitis grading power doppler",
|
||||||
|
"type": "auto",
|
||||||
|
"numResults": 10,
|
||||||
|
"includeDomains": ["pubmed.ncbi.nlm.nih.gov"],
|
||||||
|
"session_id": "sess-001"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response**
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"hits": [
|
||||||
|
{
|
||||||
|
"id": "…",
|
||||||
|
"url": "https://…",
|
||||||
|
"title": "…",
|
||||||
|
"highlights": ["…"],
|
||||||
|
"publishedDate": "…",
|
||||||
|
"score": 0.92
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"requestId": "…"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## POST /supabase/query
|
||||||
|
|
||||||
|
Allowlisted RPC only. Embedding computed server-side.
|
||||||
|
|
||||||
|
**Request**
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"rpc": "match_semantic_chunks",
|
||||||
|
"args": {
|
||||||
|
"query_text": "synovitis grade 2 knee ultrasound",
|
||||||
|
"match_count": 5,
|
||||||
|
"filter_book_ids": ["mor", "oho"]
|
||||||
|
},
|
||||||
|
"session_id": "sess-001"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response**
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"rpc": "match_semantic_chunks",
|
||||||
|
"rows": [
|
||||||
|
{
|
||||||
|
"chunk_id": "uuid",
|
||||||
|
"content": "…",
|
||||||
|
"book_id": "mor",
|
||||||
|
"parent_title": "…",
|
||||||
|
"similarity": 0.88
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Exa: https://docs.exa.ai/reference/search-api-guide-for-coding-agents
|
||||||
|
- Supabase schema: [`knowledge/spec/pg_semantic_vector_db/supabase_schema.md`](../../knowledge/spec/pg_semantic_vector_db/supabase_schema.md)
|
||||||
@@ -1,47 +1,122 @@
|
|||||||
# Backend Specification
|
# Backend Specification
|
||||||
|
|
||||||
## Purpose
|
## Code‑base Tree View (implementation)
|
||||||
Orchestrates API routers, role checks, Socratic circuit-breaker state evaluations, and coordinates ML inference, telemetry collection, and data persistence.
|
|
||||||
|
|
||||||
## Owner
|
```
|
||||||
Core Backend Team
|
backend/
|
||||||
|
├─ api/ # FastAPI routers (expose HTTP endpoints)
|
||||||
|
│ ├─ auth_api.py
|
||||||
|
│ ├─ patient_api.py
|
||||||
|
│ ├─ session_api.py
|
||||||
|
│ ├─ analysis_api.py
|
||||||
|
│ ├─ safety_api.py
|
||||||
|
│ ├─ notification_api.py
|
||||||
|
│ ├─ settings_api.py
|
||||||
|
│ ├─ ingestion_api.py
|
||||||
|
│ ├─ telemetry_api.py
|
||||||
|
├─ routers/ # Cloud LLM API routers
|
||||||
|
│ ├─ cloud_orchestrate.py # POST /api/cloud-orchestrate → Gemini proxy
|
||||||
|
│ └─ cloud_consult.py # POST /api/cloud-consult + GET /stream → MedGemma proxy
|
||||||
|
├─ services/ # Cloud LLM Gateway business logic
|
||||||
|
│ └─ cloud_llm_gateway.py # Routes Gemini/MedGemma based on task_type + consult_mode
|
||||||
|
├─ implementation/ # Deep modules (seams) – each provides a small interface
|
||||||
|
│ ├─ auth/ # Auth Module
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ └─ service.py # login, logout, refresh, me, update_me
|
||||||
|
│ ├─ patient/ # Patient Module
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ └─ service.py
|
||||||
|
│ ├─ session/ # Session & Frame handling
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ ├─ service.py
|
||||||
|
│ │ └─ frame_storage.py # S3 adapter
|
||||||
|
│ ├─ analysis_jobs/ # Async analysis orchestration
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ ├─ service.py
|
||||||
|
│ │ └─ triton_client.py # gRPC wrapper
|
||||||
|
│ ├─ safety/ # Internal safety stack
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ ├─ gradcam.py
|
||||||
|
│ │ ├─ rationale.py
|
||||||
|
│ │ ├─ circuit_breaker.py
|
||||||
|
│ │ ├─ socratic_chat.py
|
||||||
|
│ │ ├─ drift_check.py
|
||||||
|
│ │ ├─ rag_evidence.py
|
||||||
|
│ │ ├─ activations.py
|
||||||
|
│ │ └─ annotations.py
|
||||||
|
│ ├─ notification/ # Notification Module
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ └─ service.py
|
||||||
|
│ ├─ settings/ # User settings module
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ └─ service.py
|
||||||
|
│ ├─ ingestion_history/ # Ingestion history module
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ └─ service.py
|
||||||
|
│ ├─ telemetry/ # Telemetry & anomaly reporting
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ └─ service.py
|
||||||
|
│ └─ adapters/ # Low‑level adapters used by modules
|
||||||
|
│ ├─ s3_adapter.py # Generic S3 wrapper
|
||||||
|
│ ├─ triton_adapter.py # Adapter toward the Triton serving module hosted on Modal (KServe v2 HTTP / binary inference)
|
||||||
|
│ ├─ llm_adapter.py
|
||||||
|
│ └─ bert_adapter.py
|
||||||
|
└─ main.py # FastAPI app entry point, wires routers to modules
|
||||||
|
```
|
||||||
|
|
||||||
## Boundary
|
|
||||||
FastAPI server, API routers, authentication middleware, circuit breaker engine, report generator, RAG coordinator, RAG-Referee (BERT), ledger logger, and connections to Postgres + pgvector, S3 (MinIO), Redis, Triton, ladybugDB.
|
|
||||||
|
|
||||||
## Internal Design
|
|
||||||
- Built with FastAPI (Python) and Uvicorn for async HTTP server.
|
|
||||||
- Authentication middleware validates JWT tokens and enforces RBAC (roles: RO_RADIOLOGIST, RO_THERAPIST).
|
|
||||||
- Socratic circuit-breaker engine monitors interaction telemetry (hover duration, decision time, override magnitude) and triggers safety dialogs.
|
|
||||||
- Clinical Report Engine uses ReportLab to generate bilingual PDF reports per Circular 46/2018/TT-BYT.
|
|
||||||
- RAG Coordinator orchestrates Retrieval-Augmented Generation: dense vector lookup in pgvector (PostgreSQL HNSW), graph traversal in ladybugDB, mandatory pre-generation retrieval, prompt enrichment, LLM generation on browser WebLLM (GemmaE2B) or cloud Vertex AI (MedGemma via NFR-16a), and hallucination guarding via BERT RAG-Referee.
|
|
||||||
- NLP Scrubber (Microsoft Presidio): re-verifies client edge redaction, refines residual PII, and returns error if unresolvable.
|
|
||||||
- Ledger Logger appends immutable, cryptographically chained audit logs to Postgres via triggers preventing UPDATE/DELETE.
|
|
||||||
- Connections: Postgres + pgvector (via SQLAlchemy), S3 (via boto3), Redis (via redis-py), Triton (via gRPC — CV + EmbeddingGemma only), ladybugDB (via in-process C++ bindings).
|
|
||||||
- Model weights loaded at startup from internal registry; cached in memory.
|
|
||||||
- API endpoints layered: public clinical (sessions, analysis, reports, feedback) and internal/local safety (explanations, safety, drift, RAG, activations, annotations, ground-truth, escalation, morphology, telemetry).
|
|
||||||
|
|
||||||
## Interface Contract
|
## Overview
|
||||||
See `bento/backend/spec/interface-contract.md`.
|
The backend is a **FastAPI** application that orchestrates several **deep modules**. Each module presents a **seam** (the module’s public interface) that callers – the FastAPI router – use. Below is the module map, its **interface**, **implementation**, and the external **services** it depends on.
|
||||||
|
|
||||||
## Consumers
|
| Module | Interface (public API) | Implementation | External Services (dependencies) |
|
||||||
- frontend
|
|--------|------------------------|----------------|-----------------------------------|
|
||||||
|
| **Auth Module** | `login`, `logout`, `refresh`, `me`, `update_me` | JWT handling, password hashing, session store | PostgreSQL (`users` table), Redis (optional token blacklist) |
|
||||||
|
| **Patient Module** | CRUD for patients, list sessions, ingestion history | ORM models, business rules | PostgreSQL (`patients`, `sessions`, `ingestion_history`) |
|
||||||
|
| **Session Module** | Create session, add frames, retrieve, patch review | Transactional management, validation | PostgreSQL (`sessions`, `frames`), S3 adapter (frame storage) |
|
||||||
|
| **Frame Storage Adapter** | `store_frame`, `generate_presigned_url` | Modal‑based S3 client wrapper | AWS S3 (object store) |
|
||||||
|
| **Analysis Jobs Module** | `submit_job`, `job_status`, `job_steps` | Async job scheduler, Triton inference HTTP client, result aggregator | Triton inference server (KServe v2 HTTP, Modal serverless endpoint), PostgreSQL (`analysis_jobs`), S3 (artifact storage) |
|
||||||
|
| **Safety Module** | GradCAM, rationale, circuit‑breaker, Socratic chat, drift check, RAG evidence, activations, annotations, escalation, telemetry | Calls to local LLM/RAG/BERT services, post‑processing utilities | Local LLM container, BERT drift detector, RAG knowledge base, PostgreSQL (`safety_events`), S3 (heatmaps, masks) |
|
||||||
|
| **Cloud LLM Gateway Module** | `route_gemini_request`, `route_medgemma_request` | task_type matcher, NFR-16a consent/redaction/audit enforcement, consult_mode state extension, cost guarding | GCP Vertex AI (Gemini), Modal (MedGemma), Redis (consult_mode, consent, rate-limit), PostgreSQL (audit) |
|
||||||
|
| **Agent Tools Module** | `exa_search`, `supabase_query` | Exa web search proxy, Supabase allowlisted RPC, PHI-safe audit logging | Exa API, Supabase (`knowledge` schema), EmbeddingGemma (embedder TBD) |
|
||||||
|
| **Notification Module** | List, mark‑read, set preferences | Simple DB queries, push service stub | PostgreSQL (`notifications`), optional WebSocket push service |
|
||||||
|
| **Settings Module** | Get/patch user settings | DB reads/writes | PostgreSQL (`user_settings`) |
|
||||||
|
| **Ingestion History Module** | List uploads, get details | Query on `ingestion_history` table | PostgreSQL, S3 (original DICOM/frame) |
|
||||||
|
| **Telemetry Module** | Anomaly reporting | Write to telemetry tables, async queue | PostgreSQL (`telemetry`), optional analytics pipeline |
|
||||||
|
|
||||||
## Breaking-change Policy
|
## Design Principles Applied
|
||||||
See `bento/backend/spec/interface-contract.md`.
|
- **Depth**: Each module hides complex orchestration (e.g., Triton gRPC, S3 multipart upload) behind a small, well‑defined interface.
|
||||||
|
- **Seams**: Interfaces live in `backend/api/<module>_api.py`; adapters implement them in `backend/services/<module>_service.py`.
|
||||||
|
- **Deletion Test**: Removing any module concentrates its complexity inside callers, confirming the module’s value.
|
||||||
|
- **Locality**: All error handling, logging, and retry logic resides inside the module implementation, giving callers a clean contract.
|
||||||
|
- **Leverage**: Callers (FastAPI routes) only need to know request/response shapes; the module provides the full workflow.
|
||||||
|
|
||||||
## References
|
## Module Dependency Graph (Mermaid)
|
||||||
- NFR-7 (Real-Time UI Screen Refresh ≤200ms)
|
```mermaid
|
||||||
- NFR-10 (Generative Safety Guardrails)
|
graph TD;
|
||||||
- NFR-11 (Frontline Usability & Training)
|
Auth --> DB[PostgreSQL];
|
||||||
- UC-48376 (Load Patient Scan Session)
|
Patient --> DB;
|
||||||
- UC-47988 (Review Suggested Synovitis Grade)
|
Session --> DB;
|
||||||
- UC-25776 (Generate GradCAM & CoT Explanation Panel)
|
Session --> S3[AWS S3];
|
||||||
- UC-02423 (Log High-Trust Concur Block)
|
AnalysisJobs --> Triton[KServe v2 HTTP Triton = Modal Serverless];
|
||||||
- UC_Q2_* (All Quadrant 2 safety workflows)
|
AnalysisJobs --> DB;
|
||||||
- UC_Q3_* (All Quadrant 3 subservience workflows)
|
AnalysisJobs --> S3;
|
||||||
- UC_Q4_* (All Quadrant 4 double-blind workflows)
|
Safety --> LLM[Local LLM];
|
||||||
- SOFTWARE_SYSTEM_DESIGN_FR_25.md (Sections 1.2, 2.1-2.6)
|
Safety --> BERT[Drift Detector];
|
||||||
- SOLUTION_ARCHITECTURE_SPEC.md (Sections 2.1-2.6)
|
Safety --> RAG[Local RAG];
|
||||||
- DATA_ENGINEERING_SPEC.md (Sections 4-12 for domain objects)
|
Safety --> DB;
|
||||||
- CI_CD_DEPLOYMENT_PIPELINE.md (Section 9.2 for docker-compose)
|
Safety --> S3;
|
||||||
|
CloudLLM --> Gemini[GCP Vertex AI Gemini];
|
||||||
|
CloudLLM --> MedGemma[Modal MedGemma];
|
||||||
|
CloudLLM --> Redis;
|
||||||
|
CloudLLM --> DB;
|
||||||
|
CloudLLM --> CostGuard[MedGemma Usage Counter];
|
||||||
|
Notification --> DB;
|
||||||
|
Settings --> DB;
|
||||||
|
IngestionHistory --> DB;
|
||||||
|
IngestionHistory --> S3;
|
||||||
|
Telemetry --> DB;
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
*Generated for AI‑navigability and testability.*
|
||||||
@@ -1,46 +1,98 @@
|
|||||||
# Backend Interface Contract
|
# Interface Contract Catalog
|
||||||
|
|
||||||
## Purpose
|
This document enumerates every public **API contract** (HTTP endpoint) defined in `API_CONTRACT_DRAFT.md` and maps it to the **seam** (module) that fulfills it, together with the **external services** that the module interacts with.
|
||||||
Orchestrates API routers, role checks, Socratic circuit-breaker state evaluations, and coordinates ML inference, telemetry collection, and data persistence.
|
|
||||||
|
|
||||||
## Owner
|
## 0. Health & Model Registry (Infrastructure / Analysis Jobs Module)
|
||||||
Core Backend Team
|
| Method | Path | Interface Function | Dependencies |
|
||||||
|
|--------|------|-------------------|--------------|
|
||||||
|
| GET | /api/v1/health | `system.health() -> HealthStatus` | All backend dependencies |
|
||||||
|
| GET | /api/v1/model-registry | `analysis.list_registered_models() -> ModelCatalog` | Triton (Modal serverless), S3 (model artifacts) |
|
||||||
|
| POST | /api/v1/models/register | `analysis.register_model(model_id: str, file: Binary) -> RegistrationResult` | S3 (model storage), Modal (serverless provisioning) |
|
||||||
|
|
||||||
## Provides
|
## 1. Authentication Endpoints (Auth Module)
|
||||||
- api endpoints (session management, frame upload, analysis jobs, reporting, feedback, safety endpoints)
|
| Method | Path | Interface Function | Dependencies |
|
||||||
- model inference orchestration (dispatches to Triton, aggregates results)
|
|--------|------|-------------------|--------------|
|
||||||
- telemetry collection (edge-based behavioral summaries, audit logs)
|
| POST | /api/v1/auth/login | `auth.login(username: str, password: str) -> JWT` | PostgreSQL (users), bcrypt, optional Redis blacklist |
|
||||||
- data persistence coordination (writes to Postgres, S3, Redis)
|
| POST | /api/v1/auth/logout | `auth.logout(token: str) -> None` | Redis (token revocation) |
|
||||||
|
| POST | /api/v1/auth/refresh | `auth.refresh(refresh_token: str) -> JWT` | PostgreSQL, Redis |
|
||||||
|
| GET | /api/v1/users/me | `auth.me(token: str) -> UserProfile` | PostgreSQL |
|
||||||
|
| PATCH | /api/v1/users/me | `auth.update_me(token: str, updates: dict) -> UserProfile` | PostgreSQL |
|
||||||
|
|
||||||
## Consumes
|
## 2. Patient Management (Patient Module)
|
||||||
- data:storage-spec (Postgres DB, S3 object store, Redis cache)
|
| Method | Path | Interface Function | Dependencies |
|
||||||
- ml:inference-spec (Triton server for angle, inflammation, segmentation, severity)
|
|--------|------|-------------------|--------------|
|
||||||
- knowledge:guideline-spec (Qdrant vector DB, ladybugDB graph DB for grounded explanations)
|
| GET | /api/v1/patients | `patient.list(user_id: str) -> List[Patient]` | PostgreSQL |
|
||||||
|
| POST | /api/v1/patients | `patient.create(data: dict) -> Patient` | PostgreSQL |
|
||||||
|
| GET | /api/v1/patients/{patient_id} | `patient.get(id: str) -> Patient` | PostgreSQL |
|
||||||
|
| GET | /api/v1/patients/{patient_id}/sessions | `patient.list_sessions(id: str) -> List[Session]` | PostgreSQL |
|
||||||
|
| GET | /api/v1/patients/{patient_id}/history | `patient.ingestion_history(id: str) -> List[IngestionRecord]` | PostgreSQL, S3 |
|
||||||
|
|
||||||
## Consumers
|
## 3. Notification Endpoints (Notification Module)
|
||||||
- frontend
|
| Method | Path | Interface Function | Dependencies |
|
||||||
|
|--------|------|-------------------|--------------|
|
||||||
|
| GET | /api/v1/notifications | `notification.list(user_id: str, filters: dict) -> List[Notification]` | PostgreSQL |
|
||||||
|
| PATCH | /api/v1/notifications/{notification_id}/read | `notification.mark_read(id: str) -> None` | PostgreSQL |
|
||||||
|
| POST | /api/v1/notifications/preferences | `notification.set_preferences(user_id: str, prefs: dict) -> None` | PostgreSQL |
|
||||||
|
|
||||||
## Not Directly Consumable
|
## 4. Settings & Preferences (Settings Module)
|
||||||
- data internals (Postgres tables, S3 object layout, Redis keys)
|
| Method | Path | Interface Function | Dependencies |
|
||||||
- ml internals (Triton model details, GPU kernels)
|
|--------|------|-------------------|--------------|
|
||||||
- knowledge internals (Qdrant vectors, ladybugDB graph)
|
| GET | /api/v1/settings | `settings.get(user_id: str) -> Settings` | PostgreSQL |
|
||||||
|
| PATCH | /api/v1/settings | `settings.update(user_id: str, updates: dict) -> Settings` | PostgreSQL |
|
||||||
|
|
||||||
## Breaking-change Policy
|
## 5. Ingestion History (Ingestion History Module)
|
||||||
- API versioning via path (e.g., /api/v1/).
|
| Method | Path | Interface Function | Dependencies |
|
||||||
- Backward compatibility maintained for one minor version.
|
|--------|------|-------------------|--------------|
|
||||||
- Deprecation notices issued in release notes.
|
| GET | /api/v1/ingestion-history | `ingestion.list(user_id: str) -> List[Record]` | PostgreSQL, S3 |
|
||||||
- Model interface changes (input/output tensors) require version bump.
|
| GET | /api/v1/ingestion-history/{record_id} | `ingestion.get(id: str) -> RecordDetail` | PostgreSQL, S3 |
|
||||||
|
|
||||||
## References
|
## 6. Clinical Workflow Endpoints (Session & Analysis Modules)
|
||||||
- NFR-7 (Real-Time UI Screen Refresh ≤200ms)
|
| Method | Path | Interface Function | Module | Dependencies |
|
||||||
- NFR-10 (Generative Safety Guardrails)
|
|--------|------|-------------------|--------|--------------|
|
||||||
- NFR-11 (Frontline Usability & Training)
|
| POST | /api/v1/sessions | `session.create(user_id: str, patient_id: str) -> Session` | Session Module | PostgreSQL |
|
||||||
- UC-48376 (Load Patient Scan Session)
|
| GET | /api/v1/sessions/{session_id} | `session.get(id: str) -> SessionDetail` | Session Module | PostgreSQL |
|
||||||
- UC-47988 (Review Suggested Synovitis Grade)
|
| POST | /api/v1/sessions/{session_id}/frames | `session.add_frame(id: str, file: UploadFile) -> FrameMeta` | Session Module (via Frame Storage Adapter) | S3, PostgreSQL |
|
||||||
- UC-25776 (Generate GradCAM & CoT Explanation Panel)
|
| PATCH | /api/v1/sessions/{session_id}/review | `session.patch_review(id: str, review: dict) -> Session` | Session Module | PostgreSQL |
|
||||||
- UC-02423 (Log High-Trust Concur Block)
|
| POST | /api/v1/analysis-jobs | `analysis.submit(session_id: str, params: dict) -> JobID` | Analysis Jobs Module | Triton, PostgreSQL, S3 |
|
||||||
- UC_Q2_* (All Quadrant 2 safety workflows)
|
| GET | /api/v1/analysis-jobs/{job_id} | `analysis.status(job_id: str) -> JobStatus` | Analysis Jobs Module | PostgreSQL |
|
||||||
- UC_Q3_* (All Quadrant 3 subservience workflows)
|
| GET | /api/v1/analysis-jobs/{job_id}/steps | `analysis.steps(job_id: str) -> List[Step]` | Analysis Jobs Module | PostgreSQL |
|
||||||
- UC_Q4_* (All Quadrant 4 double-blind workflows)
|
| POST | /api/v1/reports | `report.create(session_id: str, payload: dict) -> ReportID` | Session Module | PostgreSQL, S3 |
|
||||||
- SOFTWARE_SYSTEM_DESIGN_FR_25.md (Sections 1.2, 2.1-2.6)
|
| POST | /api/v1/reports/{report_id}/sign | `report.sign(id: str, signature: dict) -> Report` | Session Module | PostgreSQL |
|
||||||
- SOLUTION_ARCHITECTURE_SPEC.md (Sections 2.1-2.6)
|
| POST | /api/v1/reports/{report_id}/emr-sync | `report.sync_emr(id: str) -> SyncResult` | Session Module | External EMR connector (REST) |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/feedback | `safety.submit_correction(session_id: str, correction: dict) -> None` | Safety Module | PostgreSQL, async analytics pipeline |
|
||||||
|
| POST | /api/v1/analysis | `analysis.submit_sync(session_id: str, params: dict) -> JobResult` | Analysis Jobs Module | Triton, PostgreSQL, S3 |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/persist | `session.persist(session_id: str, review: dict) -> PersistResult` | Session Module | PostgreSQL, S3 |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/export-pdf | `session.export_pdf(session_id: str, params: dict) -> ExportResult` | Session Module | S3 |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/scrub-validate | `session.scrub_validate(session_id: str, metadata: dict) -> ScrubResult` | Session Module | - |
|
||||||
|
| GET | /api/v1/analysis-jobs/{job_id}/stream | `analysis.stream(job_id: str) -> SSE[StepEvent]` | Analysis Jobs Module | - |
|
||||||
|
|
||||||
|
## 8. Cloud LLM Orchestration (Cloud LLM Gateway)
|
||||||
|
| Method | Path | Interface Function | Dependencies |
|
||||||
|
|--------|------|-------------------|--------------|
|
||||||
|
| POST | /api/v1/cloud-orchestrate | `cloud_llm_gateway.route_gemini_request(payload, user_id) -> dict` | Vertex AI (Gemini), Redis (consult_mode, consent), audit log |
|
||||||
|
| POST | /api/v1/cloud-consult | `cloud_llm_gateway.route_medgemma_request(payload, user_id) -> dict` | Modal MedGemma, Redis (consult_mode, consent), audit log |
|
||||||
|
| GET | /api/v1/cloud-consult/stream | `cloud_llm_gateway.route_medgemma_request(payload, user_id) -> SSE[chunk]` | Modal MedGemma streaming, Redis, audit log |
|
||||||
|
|
||||||
|
## 9. Internal/Local Safety Endpoints (Safety Module)
|
||||||
|
| Method | Path | Interface Function | Dependencies |
|
||||||
|
|--------|------|-------------------|--------------|
|
||||||
|
| POST | /api/v1/sessions/{session_id}/explanations/gradcam | `safety.gradcam(session_id: str) -> HeatmapURL` | Triton (model output), S3 (store heatmap) |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/explanations/rationale | `safety.rationale(session_id: str) -> Text` | Local LLM service |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/safety/circuit-breaker | `safety.circuit_break(session_id: str, flag: bool) -> None` | PostgreSQL |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/chat/socratic | `safety.socratic_chat(session_id: str, prompt: str) -> ChatResponse` | Local LLM, PostgreSQL |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/drift/check | `safety.drift_check(session_id: str) -> DriftResult` | BERT (Drift Detector) |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/rag/evidence | `safety.rag_evidence(session_id: str) -> EvidenceList` | Local RAG |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/activations | `safety.activations(session_id: str, params: dict) -> ActivationMeta` | Triton, S3 |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/annotations/artifacts | `safety.upload_artifact(session_id: str, file: UploadFile) -> ArtifactMeta` | S3 |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/ground-truth | `safety.ground_truth(session_id: str, label: dict) -> None` | PostgreSQL |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/escalation | `safety.escalate(session_id: str, reason: str) -> EscalationTicket` | PostgreSQL, external ticketing stub |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/annotations/morphology | `safety.morphology(session_id: str, annotation: dict) -> None` | PostgreSQL |
|
||||||
|
| POST | /api/v1/sessions/{session_id}/telemetry/anomalies | `telemetry.anomaly(session_id: str, data: dict) -> None` | PostgreSQL, async analytics pipeline |
|
||||||
|
| POST | /api/v1/safety/guardrail-check | `safety.guardrail_check(session_id: str, prompt: str, score: float) -> GuardrailResult` | Safety Module | - |
|
||||||
|
| GET | /api/v1/sessions/{session_id}/chat/stream | `safety.chat_stream(session_id: str) -> SSE[ChatEvent]` | Safety Module | - |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Notation**: each row describes the **seam** (module) that implements the endpoint. Callers only need to know the request/response signature; the module encapsulates all orchestration, giving high **leverage** and good **testability**.
|
||||||
|
|
||||||
|
*Generated to aid AI‑navigability and automated test generation.*
|
||||||
@@ -0,0 +1,277 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Stratified sampling of test_images into per-patient scan profiles.
|
||||||
|
|
||||||
|
Each patient receives the same number of frames: (images_per_stratum × 5 strata).
|
||||||
|
Strata = folder names under backend/tests/test_images/:
|
||||||
|
- sup-up-long_positive
|
||||||
|
- sup-up-long_negative
|
||||||
|
- post_trans_positive
|
||||||
|
- post_trans_negative
|
||||||
|
- other_angle
|
||||||
|
|
||||||
|
Sampling is with replacement within each stratum (per patient).
|
||||||
|
|
||||||
|
Outputs:
|
||||||
|
- backend/tests/test_images/profiles/manifest.json
|
||||||
|
- frontend/implementation/public/assets/patient-profiles/{patient_id}/...
|
||||||
|
- frontend/implementation/src/data/patientScanProfiles.generated.ts
|
||||||
|
|
||||||
|
Run from CODEBASE root:
|
||||||
|
|
||||||
|
python backend/tests/sample_patient_profiles.py
|
||||||
|
python backend/tests/sample_patient_profiles.py --per-stratum 2 --seed 42
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import random
|
||||||
|
import shutil
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
CODEBASE_ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
TEST_IMAGES_ROOT = CODEBASE_ROOT / "backend/tests/test_images"
|
||||||
|
PROFILES_MANIFEST = TEST_IMAGES_ROOT / "profiles" / "manifest.json"
|
||||||
|
PUBLIC_PROFILES_ROOT = CODEBASE_ROOT / "frontend/implementation/public/assets/patient-profiles"
|
||||||
|
GENERATED_TS = CODEBASE_ROOT / "frontend/implementation/src/data/patientScanProfiles.generated.ts"
|
||||||
|
|
||||||
|
IMAGE_SUFFIXES = {".png", ".jpg", ".jpeg", ".webp", ".bmp"}
|
||||||
|
|
||||||
|
STRATA: list[str] = [
|
||||||
|
"sup-up-long_positive",
|
||||||
|
"sup-up-long_negative",
|
||||||
|
"post_trans_positive",
|
||||||
|
"post_trans_negative",
|
||||||
|
"other_angle",
|
||||||
|
]
|
||||||
|
|
||||||
|
STRATUM_SLUG: dict[str, str] = {
|
||||||
|
"sup-up-long_positive": "sup-long-pos",
|
||||||
|
"sup-up-long_negative": "sup-long-neg",
|
||||||
|
"post_trans_positive": "post-trans-pos",
|
||||||
|
"post_trans_negative": "post-trans-neg",
|
||||||
|
"other_angle": "other",
|
||||||
|
}
|
||||||
|
|
||||||
|
STRATUM_LABEL_VI: dict[str, str] = {
|
||||||
|
"sup-up-long_positive": "Sup dọc — viêm (+)",
|
||||||
|
"sup-up-long_negative": "Sup dọc — không viêm (−)",
|
||||||
|
"post_trans_positive": "Sau ngang — viêm (+)",
|
||||||
|
"post_trans_negative": "Sau ngang — không viêm (−)",
|
||||||
|
"other_angle": "Góc khác (med-lat / sup-trans-flex)",
|
||||||
|
}
|
||||||
|
|
||||||
|
EXPECTED_ANGLE_BY_STRATUM: dict[str, str | None] = {
|
||||||
|
"sup-up-long_positive": "sup-up-long",
|
||||||
|
"sup-up-long_negative": "sup-up-long",
|
||||||
|
"post_trans_positive": "post-trans",
|
||||||
|
"post_trans_negative": "post-trans",
|
||||||
|
"other_angle": None,
|
||||||
|
}
|
||||||
|
|
||||||
|
PATIENTS: list[dict[str, str]] = [
|
||||||
|
{"id": "p-001", "name": "Nguyễn Văn An", "mrn": "BN-2024-1847"},
|
||||||
|
{"id": "p-002", "name": "Trần Thị Bích", "mrn": "BN-2024-1923"},
|
||||||
|
{"id": "p-003", "name": "Lê Hoàng Minh", "mrn": "BN-2024-2011"},
|
||||||
|
{"id": "p-004", "name": "Phạm Thu Hà", "mrn": "BN-2024-2088"},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PoolImage:
|
||||||
|
path: Path
|
||||||
|
stratum: str
|
||||||
|
|
||||||
|
|
||||||
|
def _list_images(folder: Path) -> list[Path]:
|
||||||
|
if not folder.is_dir():
|
||||||
|
return []
|
||||||
|
files = [
|
||||||
|
p
|
||||||
|
for p in sorted(folder.iterdir())
|
||||||
|
if p.is_file() and p.suffix.lower() in IMAGE_SUFFIXES
|
||||||
|
]
|
||||||
|
return files
|
||||||
|
|
||||||
|
|
||||||
|
def _infer_other_angle_class(filename: str) -> str:
|
||||||
|
lower = filename.lower()
|
||||||
|
if "trans" in lower and "flex" in lower:
|
||||||
|
return "sup-trans-flex"
|
||||||
|
if "med" in lower and "lat" in lower:
|
||||||
|
return "med-lat"
|
||||||
|
return "med-lat"
|
||||||
|
|
||||||
|
|
||||||
|
def _build_pools() -> dict[str, list[PoolImage]]:
|
||||||
|
pools: dict[str, list[PoolImage]] = {}
|
||||||
|
for stratum in STRATA:
|
||||||
|
folder = TEST_IMAGES_ROOT / stratum
|
||||||
|
images = _list_images(folder)
|
||||||
|
if not images:
|
||||||
|
raise FileNotFoundError(f"No images in stratum folder: {folder}")
|
||||||
|
pools[stratum] = [PoolImage(path=p, stratum=stratum) for p in images]
|
||||||
|
return pools
|
||||||
|
|
||||||
|
|
||||||
|
def _sample_profile(
|
||||||
|
patient: dict[str, str],
|
||||||
|
pools: dict[str, list[PoolImage]],
|
||||||
|
*,
|
||||||
|
per_stratum: int,
|
||||||
|
rng: random.Random,
|
||||||
|
) -> list[dict]:
|
||||||
|
frames: list[dict] = []
|
||||||
|
for stratum in STRATA:
|
||||||
|
pool = pools[stratum]
|
||||||
|
slug = STRATUM_SLUG[stratum]
|
||||||
|
for index in range(per_stratum):
|
||||||
|
chosen = rng.choice(pool)
|
||||||
|
source_name = chosen.path.name
|
||||||
|
expected = EXPECTED_ANGLE_BY_STRATUM[stratum]
|
||||||
|
if expected is None:
|
||||||
|
expected = _infer_other_angle_class(source_name)
|
||||||
|
|
||||||
|
frame_id = f"{patient['id']}-{slug}-{index}"
|
||||||
|
ext = chosen.path.suffix.lower()
|
||||||
|
asset_name = f"{slug}-{index}{ext}"
|
||||||
|
rel_asset = f"/assets/patient-profiles/{patient['id']}/{asset_name}"
|
||||||
|
|
||||||
|
frames.append(
|
||||||
|
{
|
||||||
|
"id": frame_id,
|
||||||
|
"patient_id": patient["id"],
|
||||||
|
"stratum": stratum,
|
||||||
|
"stratum_index": index,
|
||||||
|
"label": f"{STRATUM_LABEL_VI[stratum]} · #{index + 1}",
|
||||||
|
"expected_angle_class": expected,
|
||||||
|
"source_path": str(chosen.path.relative_to(CODEBASE_ROOT)),
|
||||||
|
"source_filename": source_name,
|
||||||
|
"asset_path": rel_asset,
|
||||||
|
"asset_filename": asset_name,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return frames
|
||||||
|
|
||||||
|
|
||||||
|
def _materialize_assets(patient_id: str, frames: list[dict]) -> None:
|
||||||
|
out_dir = PUBLIC_PROFILES_ROOT / patient_id
|
||||||
|
if out_dir.exists():
|
||||||
|
shutil.rmtree(out_dir)
|
||||||
|
out_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
for frame in frames:
|
||||||
|
src = CODEBASE_ROOT / frame["source_path"]
|
||||||
|
dst = out_dir / frame["asset_filename"]
|
||||||
|
shutil.copy2(src, dst)
|
||||||
|
|
||||||
|
|
||||||
|
def _write_manifest(payload: dict) -> None:
|
||||||
|
PROFILES_MANIFEST.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
PROFILES_MANIFEST.write_text(
|
||||||
|
json.dumps(payload, indent=2, ensure_ascii=False) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _ts_string(value: str) -> str:
|
||||||
|
return json.dumps(value, ensure_ascii=False)
|
||||||
|
|
||||||
|
|
||||||
|
def _write_generated_ts(payload: dict) -> None:
|
||||||
|
lines = [
|
||||||
|
"/** Auto-generated by backend/tests/sample_patient_profiles.py — do not edit. */",
|
||||||
|
"import type { ScanFrame } from './scanFrames';",
|
||||||
|
"",
|
||||||
|
"export interface PatientScanProfileFrame extends ScanFrame {",
|
||||||
|
" stratum: string;",
|
||||||
|
" expectedAngleClass?: string;",
|
||||||
|
" sourcePath: string;",
|
||||||
|
"}",
|
||||||
|
"",
|
||||||
|
"export const PATIENT_SCAN_PROFILES: Record<string, PatientScanProfileFrame[]> = {",
|
||||||
|
]
|
||||||
|
|
||||||
|
for patient in payload["patients"]:
|
||||||
|
pid = patient["id"]
|
||||||
|
lines.append(f" {_ts_string(pid)}: [")
|
||||||
|
for frame in patient["frames"]:
|
||||||
|
lines.append(
|
||||||
|
" {"
|
||||||
|
f" id: {_ts_string(frame['id'])},"
|
||||||
|
f" src: {_ts_string(frame['asset_path'])},"
|
||||||
|
f" label: {_ts_string(frame['label'])},"
|
||||||
|
f" stratum: {_ts_string(frame['stratum'])},"
|
||||||
|
f" expectedAngleClass: {_ts_string(frame['expected_angle_class'])},"
|
||||||
|
f" sourcePath: {_ts_string(frame['source_path'])},"
|
||||||
|
" },"
|
||||||
|
)
|
||||||
|
lines.append(" ],")
|
||||||
|
|
||||||
|
lines.extend(
|
||||||
|
[
|
||||||
|
"};",
|
||||||
|
"",
|
||||||
|
"export function getScanFramesForPatient(patientId: string): PatientScanProfileFrame[] {",
|
||||||
|
" return PATIENT_SCAN_PROFILES[patientId] ?? PATIENT_SCAN_PROFILES['p-001'] ?? [];",
|
||||||
|
"}",
|
||||||
|
"",
|
||||||
|
f"export const FRAMES_PER_PATIENT = {payload['frames_per_patient']};",
|
||||||
|
f"export const IMAGES_PER_STRATUM = {payload['images_per_stratum']};",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
GENERATED_TS.write_text("\n".join(lines), encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description="Sample stratified ultrasound frames per patient profile.")
|
||||||
|
parser.add_argument(
|
||||||
|
"--per-stratum",
|
||||||
|
type=int,
|
||||||
|
default=1,
|
||||||
|
help="Images sampled per stratum folder per patient (default: 1 → 5 frames/patient).",
|
||||||
|
)
|
||||||
|
parser.add_argument("--seed", type=int, default=2026, help="RNG seed for reproducible sampling.")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if args.per_stratum < 1:
|
||||||
|
raise SystemExit("--per-stratum must be >= 1")
|
||||||
|
|
||||||
|
rng = random.Random(args.seed)
|
||||||
|
pools = _build_pools()
|
||||||
|
|
||||||
|
print("Stratum pool sizes:")
|
||||||
|
for stratum in STRATA:
|
||||||
|
print(f" {stratum}: {len(pools[stratum])} images")
|
||||||
|
|
||||||
|
profiles: list[dict] = []
|
||||||
|
for patient in PATIENTS:
|
||||||
|
frames = _sample_profile(patient, pools, per_stratum=args.per_stratum, rng=rng)
|
||||||
|
_materialize_assets(patient["id"], frames)
|
||||||
|
profiles.append({**patient, "frames": frames})
|
||||||
|
print(f"Patient {patient['id']}: {len(frames)} frames")
|
||||||
|
|
||||||
|
frames_per_patient = args.per_stratum * len(STRATA)
|
||||||
|
payload = {
|
||||||
|
"seed": args.seed,
|
||||||
|
"images_per_stratum": args.per_stratum,
|
||||||
|
"frames_per_patient": frames_per_patient,
|
||||||
|
"strata": STRATA,
|
||||||
|
"patients": profiles,
|
||||||
|
}
|
||||||
|
|
||||||
|
_write_manifest(payload)
|
||||||
|
_write_generated_ts(payload)
|
||||||
|
|
||||||
|
print(f"\nWrote manifest: {PROFILES_MANIFEST.relative_to(CODEBASE_ROOT)}")
|
||||||
|
print(f"Wrote assets: {PUBLIC_PROFILES_ROOT.relative_to(CODEBASE_ROOT)}/")
|
||||||
|
print(f"Wrote TS: {GENERATED_TS.relative_to(CODEBASE_ROOT)}")
|
||||||
|
print(f"Total: {len(PATIENTS)} patients × {frames_per_patient} frames = {len(PATIENTS) * frames_per_patient} images")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
"""
|
||||||
|
Minimal BFF for agent-tool smoke tests (no GCP secrets, no Redis).
|
||||||
|
|
||||||
|
Mounts only:
|
||||||
|
POST /api/v1/embed
|
||||||
|
POST /api/v1/agent/tools/exa/search
|
||||||
|
POST /api/v1/agent/tools/supabase/query
|
||||||
|
|
||||||
|
Run from CODEBASE root:
|
||||||
|
# loads PILOT_PROJECT/secrets/aws_secret/.env if present
|
||||||
|
PYTHONPATH=. python backend/tests/smoke_agent_tools_server.py
|
||||||
|
|
||||||
|
Then:
|
||||||
|
cd ml/tests/agent_tools && npm run smoke:bff
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import uvicorn
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
|
||||||
|
CODEBASE_ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
if str(CODEBASE_ROOT) not in sys.path:
|
||||||
|
sys.path.insert(0, str(CODEBASE_ROOT))
|
||||||
|
|
||||||
|
SECRETS_ENV = CODEBASE_ROOT.parents[2] / "secrets" / "aws_secret" / ".env"
|
||||||
|
|
||||||
|
|
||||||
|
def _load_dotenv_file(path: Path) -> None:
|
||||||
|
if not path.exists():
|
||||||
|
return
|
||||||
|
for line in path.read_text(encoding="utf-8").splitlines():
|
||||||
|
line = line.strip()
|
||||||
|
if not line or line.startswith("#") or "=" not in line:
|
||||||
|
continue
|
||||||
|
key, _, value = line.partition("=")
|
||||||
|
key = key.strip()
|
||||||
|
value = value.strip().strip('"').strip("'")
|
||||||
|
os.environ.setdefault(key, value)
|
||||||
|
|
||||||
|
|
||||||
|
_load_dotenv_file(SECRETS_ENV)
|
||||||
|
os.environ.setdefault("EMBED_QUERY_MOCK", "1")
|
||||||
|
|
||||||
|
from backend.routers import agent_tools # noqa: E402
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
app = FastAPI(title="Agent Tools Smoke BFF", version="0.1.0")
|
||||||
|
app.add_middleware(
|
||||||
|
CORSMiddleware,
|
||||||
|
allow_origins=["*"],
|
||||||
|
allow_credentials=True,
|
||||||
|
allow_methods=["*"],
|
||||||
|
allow_headers=["*"],
|
||||||
|
)
|
||||||
|
app.include_router(agent_tools.router)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
host = os.getenv("SMOKE_BFF_HOST", "127.0.0.1")
|
||||||
|
port = int(os.getenv("SMOKE_BFF_PORT", "8000"))
|
||||||
|
logging.basicConfig(level=logging.INFO)
|
||||||
|
logger.info("Agent tools smoke BFF on http://%s:%s", host, port)
|
||||||
|
logger.info("Secrets env: %s (%s)", SECRETS_ENV, "found" if SECRETS_ENV.exists() else "missing")
|
||||||
|
uvicorn.run(app, host=host, port=port, log_level="info")
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
"""HTTP layer tests for the CV inference FastAPI router."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import AsyncMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
CODEBASE_ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
sys.path.insert(0, str(CODEBASE_ROOT))
|
||||||
|
|
||||||
|
from backend.cv_inference_server import create_app
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client() -> TestClient:
|
||||||
|
return TestClient(create_app())
|
||||||
|
|
||||||
|
|
||||||
|
def _png_bytes() -> bytes:
|
||||||
|
buf = io.BytesIO()
|
||||||
|
Image.new("RGB", (32, 24), color=(100, 120, 140)).save(buf, format="PNG")
|
||||||
|
return buf.getvalue()
|
||||||
|
|
||||||
|
|
||||||
|
def test_health_route(client: TestClient):
|
||||||
|
with patch(
|
||||||
|
"backend.routers.cv_inference.TritonAdapter.model_ready",
|
||||||
|
new=AsyncMock(return_value=True),
|
||||||
|
):
|
||||||
|
response = client.get("/api/test/health")
|
||||||
|
assert response.status_code == 200
|
||||||
|
body = response.json()
|
||||||
|
assert body["service"] == "cv-inference"
|
||||||
|
assert body["status"] == "ok"
|
||||||
|
|
||||||
|
|
||||||
|
def test_analyze_batch_route(client: TestClient):
|
||||||
|
mock_result = type(
|
||||||
|
"Batch",
|
||||||
|
(),
|
||||||
|
{
|
||||||
|
"results": [{"success": True, "frame_id": "f1", "angle": {"class": "med-lat"}}],
|
||||||
|
"triton_infer_calls": 1,
|
||||||
|
"triton_infer_modes": ["angle:batched"],
|
||||||
|
},
|
||||||
|
)()
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"backend.routers.cv_inference.run_batch",
|
||||||
|
new=AsyncMock(return_value=mock_result),
|
||||||
|
):
|
||||||
|
response = client.post(
|
||||||
|
"/api/test/analyze/batch",
|
||||||
|
data={"frame_ids": json.dumps(["f1"])},
|
||||||
|
files=[("images", ("f1.png", _png_bytes(), "image/png"))],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
body = response.json()
|
||||||
|
assert body["success"] is True
|
||||||
|
assert body["image_count"] == 1
|
||||||
|
assert body["results"][0]["frame_id"] == "f1"
|
||||||
|
|
||||||
|
|
||||||
|
def test_legacy_segment_route_returns_410(client: TestClient):
|
||||||
|
response = client.post(
|
||||||
|
"/api/test/segment/batch",
|
||||||
|
data={"frame_ids": json.dumps(["f1"]), "angle_type": "sup"},
|
||||||
|
files=[("images", ("f1.png", _png_bytes(), "image/png"))],
|
||||||
|
)
|
||||||
|
assert response.status_code == 410
|
||||||
@@ -0,0 +1,255 @@
|
|||||||
|
"""Tests for backend.services.cv_inference_service — structure, gating, cache keys."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import AsyncMock, patch
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import pytest
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
CODEBASE_ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
sys.path.insert(0, str(CODEBASE_ROOT))
|
||||||
|
|
||||||
|
MANIFEST_PATH = Path(__file__).resolve().parent / "test_images" / "profiles" / "manifest.json"
|
||||||
|
|
||||||
|
REQUIRED_RESULT_KEYS = {
|
||||||
|
"success",
|
||||||
|
"angle",
|
||||||
|
"segmentation",
|
||||||
|
"severity",
|
||||||
|
"images",
|
||||||
|
"models_used",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _load_manifest_frames() -> list[dict]:
|
||||||
|
if not MANIFEST_PATH.exists():
|
||||||
|
return []
|
||||||
|
data = json.loads(MANIFEST_PATH.read_text(encoding="utf-8"))
|
||||||
|
frames: list[dict] = []
|
||||||
|
for patient in data.get("patients", []):
|
||||||
|
frames.extend(patient.get("frames", []))
|
||||||
|
return frames
|
||||||
|
|
||||||
|
|
||||||
|
def _frame_image_path(frame: dict) -> Path:
|
||||||
|
return CODEBASE_ROOT / frame["source_path"]
|
||||||
|
|
||||||
|
|
||||||
|
def _make_angle_logits(class_index: int, num_classes: int = 4) -> np.ndarray:
|
||||||
|
row = np.full(num_classes, -2.0, dtype=np.float32)
|
||||||
|
row[class_index] = 5.0
|
||||||
|
return row
|
||||||
|
|
||||||
|
|
||||||
|
ANGLE_CLASS_INDEX = {
|
||||||
|
"med-lat": 0,
|
||||||
|
"post-trans": 1,
|
||||||
|
"sup-trans-flex": 2,
|
||||||
|
"sup-up-long": 3,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def sample_rgb_image() -> Image.Image:
|
||||||
|
return Image.new("RGB", (128, 96), color=(80, 120, 160))
|
||||||
|
|
||||||
|
|
||||||
|
def test_analyze_batch_cache_key_stable_order():
|
||||||
|
from backend.services.cv_result_cache import analyze_batch_cache_key
|
||||||
|
|
||||||
|
key_a = analyze_batch_cache_key(
|
||||||
|
["frame-b", "frame-a"],
|
||||||
|
["hash-b", "hash-a"],
|
||||||
|
)
|
||||||
|
key_b = analyze_batch_cache_key(
|
||||||
|
["frame-a", "frame-b"],
|
||||||
|
["hash-a", "hash-b"],
|
||||||
|
)
|
||||||
|
assert key_a == key_b
|
||||||
|
assert key_a.startswith("analyze|")
|
||||||
|
|
||||||
|
|
||||||
|
def test_cv_result_cache_coalesces_inflight():
|
||||||
|
from backend.services import cv_result_cache
|
||||||
|
|
||||||
|
calls = 0
|
||||||
|
|
||||||
|
async def slow_compute():
|
||||||
|
nonlocal calls
|
||||||
|
calls += 1
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
async def run():
|
||||||
|
return await asyncio.gather(
|
||||||
|
cv_result_cache.with_result_cache("test-key-coalesce", slow_compute),
|
||||||
|
cv_result_cache.with_result_cache("test-key-coalesce", slow_compute),
|
||||||
|
)
|
||||||
|
|
||||||
|
results = asyncio.run(run())
|
||||||
|
assert results == [{"ok": True}, {"ok": True}]
|
||||||
|
assert calls == 1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"angle_class,inflammation_detected,expect_seg_performed",
|
||||||
|
[
|
||||||
|
("med-lat", False, False),
|
||||||
|
("sup-trans-flex", False, False),
|
||||||
|
("post-trans", False, False),
|
||||||
|
("post-trans", True, True),
|
||||||
|
("sup-up-long", False, False),
|
||||||
|
("sup-up-long", True, True),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_run_single_gating_logic(
|
||||||
|
sample_rgb_image: Image.Image,
|
||||||
|
angle_class: str,
|
||||||
|
inflammation_detected: bool,
|
||||||
|
expect_seg_performed: bool,
|
||||||
|
):
|
||||||
|
from backend.services.cv_inference_service import CvInferenceOptions, run_single
|
||||||
|
|
||||||
|
angle_idx = ANGLE_CLASS_INDEX[angle_class]
|
||||||
|
inflam_logits = _make_angle_logits(1 if inflammation_detected else 0, num_classes=2)
|
||||||
|
seg_logits = np.zeros((1, 7, 64, 64), dtype=np.float32)
|
||||||
|
|
||||||
|
async def mock_angle_single(image, model_name):
|
||||||
|
return _make_angle_logits(angle_idx), "angle:batched", 1
|
||||||
|
|
||||||
|
async def mock_inflam_single(image, model_name):
|
||||||
|
return inflam_logits, "inflam:batched", 1
|
||||||
|
|
||||||
|
async def mock_seg_single(image, model_name):
|
||||||
|
return seg_logits[0], "seg:batched", 1
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch(
|
||||||
|
"backend.services.cv_inference_service.triton_runtime.infer_angle_logits_single",
|
||||||
|
new=AsyncMock(side_effect=mock_angle_single),
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"backend.services.cv_inference_service.triton_runtime.infer_inflammation_logits_single",
|
||||||
|
new=AsyncMock(side_effect=mock_inflam_single),
|
||||||
|
),
|
||||||
|
patch(
|
||||||
|
"backend.services.cv_inference_service.triton_runtime.infer_segmentation_logits_single",
|
||||||
|
new=AsyncMock(side_effect=mock_seg_single),
|
||||||
|
),
|
||||||
|
):
|
||||||
|
result = asyncio.run(
|
||||||
|
run_single(
|
||||||
|
sample_rgb_image,
|
||||||
|
frame_id="test-frame",
|
||||||
|
options=CvInferenceOptions(use_cache=False),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["success"] is True
|
||||||
|
assert REQUIRED_RESULT_KEYS.issubset(result.keys())
|
||||||
|
assert result["angle"]["class"] == angle_class
|
||||||
|
assert result["segmentation"]["performed"] is expect_seg_performed
|
||||||
|
if expect_seg_performed:
|
||||||
|
assert "segmented" in result["images"]
|
||||||
|
assert "inflammation" in result["models_used"]
|
||||||
|
assert "segmentation" in result["models_used"]
|
||||||
|
elif angle_class in {"post-trans", "sup-up-long"}:
|
||||||
|
assert result["inflammation"]["detected"] is inflammation_detected
|
||||||
|
assert result["severity"]["level"] == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_batch_result_shape(sample_rgb_image: Image.Image):
|
||||||
|
from backend.services.cv_inference_service import CvInferenceOptions, run_batch
|
||||||
|
|
||||||
|
async def mock_angle_single(image, model_name):
|
||||||
|
return _make_angle_logits(ANGLE_CLASS_INDEX["med-lat"]), "angle:batched", 1
|
||||||
|
|
||||||
|
with patch(
|
||||||
|
"backend.services.cv_inference_service.triton_runtime.infer_angle_logits_single",
|
||||||
|
new=AsyncMock(side_effect=mock_angle_single),
|
||||||
|
):
|
||||||
|
batch = asyncio.run(
|
||||||
|
run_batch(
|
||||||
|
[sample_rgb_image, sample_rgb_image],
|
||||||
|
["f1", "f2"],
|
||||||
|
options=CvInferenceOptions(use_cache=False),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(batch.results) == 2
|
||||||
|
assert batch.triton_infer_calls == 2
|
||||||
|
assert len(batch.triton_infer_modes) == 2
|
||||||
|
for item in batch.results:
|
||||||
|
assert REQUIRED_RESULT_KEYS.issubset(item.keys())
|
||||||
|
assert item["success"] is True
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.skipif(
|
||||||
|
not os.getenv("RUN_CV_INTEGRATION"),
|
||||||
|
reason="Set RUN_CV_INTEGRATION=1 to call live Modal Triton",
|
||||||
|
)
|
||||||
|
def test_run_single_live_other_angle_frame():
|
||||||
|
frames = _load_manifest_frames()
|
||||||
|
other_frames = [f for f in frames if f.get("stratum") == "other_angle"]
|
||||||
|
if not other_frames:
|
||||||
|
pytest.skip("No other_angle frames in manifest")
|
||||||
|
|
||||||
|
frame = other_frames[0]
|
||||||
|
image_path = _frame_image_path(frame)
|
||||||
|
if not image_path.exists():
|
||||||
|
pytest.skip(f"Test image missing: {image_path}")
|
||||||
|
|
||||||
|
from backend.services.cv_inference_service import CvInferenceOptions, run_single
|
||||||
|
|
||||||
|
image = Image.open(image_path).convert("RGB")
|
||||||
|
result = asyncio.run(
|
||||||
|
run_single(
|
||||||
|
image,
|
||||||
|
frame_id=frame["id"],
|
||||||
|
options=CvInferenceOptions(use_cache=False),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["success"] is True
|
||||||
|
assert REQUIRED_RESULT_KEYS.issubset(result.keys())
|
||||||
|
assert result["segmentation"]["performed"] is False
|
||||||
|
assert result["severity"]["level"] == 0
|
||||||
|
assert "enhanced" in result["images"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.skipif(
|
||||||
|
not os.getenv("RUN_CV_INTEGRATION"),
|
||||||
|
reason="Set RUN_CV_INTEGRATION=1 to call live Modal Triton",
|
||||||
|
)
|
||||||
|
def test_run_batch_live_from_manifest():
|
||||||
|
frames = _load_manifest_frames()
|
||||||
|
if not frames:
|
||||||
|
pytest.skip("Manifest not found or empty")
|
||||||
|
|
||||||
|
selected = frames[:2]
|
||||||
|
images: list[Image.Image] = []
|
||||||
|
frame_ids: list[str] = []
|
||||||
|
for frame in selected:
|
||||||
|
path = _frame_image_path(frame)
|
||||||
|
if not path.exists():
|
||||||
|
pytest.skip(f"Test image missing: {path}")
|
||||||
|
images.append(Image.open(path).convert("RGB"))
|
||||||
|
frame_ids.append(frame["id"])
|
||||||
|
|
||||||
|
from backend.services.cv_inference_service import CvInferenceOptions, run_batch
|
||||||
|
|
||||||
|
batch = asyncio.run(
|
||||||
|
run_batch(images, frame_ids, options=CvInferenceOptions(use_cache=False))
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(batch.results) == len(images)
|
||||||
|
assert batch.triton_infer_calls >= len(images)
|
||||||
|
for item in batch.results:
|
||||||
|
assert item["success"] is True
|
||||||
|
assert REQUIRED_RESULT_KEYS.issubset(item.keys())
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
"""
|
||||||
|
Backward-compatible launcher for the CV inference FastAPI service.
|
||||||
|
|
||||||
|
Prefer:
|
||||||
|
|
||||||
|
PYTHONPATH=. python -m backend.cv_inference_server
|
||||||
|
|
||||||
|
This module remains so existing docs/scripts that invoke
|
||||||
|
`backend/tests/test_fast_api_proxy.py` keep working.
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
|
||||||
|
os.environ.setdefault(
|
||||||
|
"TRITON_ENDPOINT",
|
||||||
|
"https://dtj-tran--triton-s3-service-unified-triton-server.modal.run",
|
||||||
|
)
|
||||||
|
|
||||||
|
from backend.cv_inference_server import main
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 85 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 109 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 98 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 122 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 95 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 138 KiB |
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user