update the cv modal inference proxy server with optimization #6
6
.gitignore
vendored
6
.gitignore
vendored
@@ -227,7 +227,6 @@ Policy_Analysis/
|
||||
package-lock.json
|
||||
package.json
|
||||
.env.development
|
||||
dist/
|
||||
run.sh
|
||||
workspace/sprint_1_2/CODEBASE/knowledge/implementation/model/
|
||||
*.onnx_data
|
||||
@@ -236,3 +235,8 @@ workspace/sprint_1_2/CODEBASE/knowledge/implementation/model/
|
||||
*.safetensors
|
||||
*.pt
|
||||
*.pth
|
||||
*.pdf
|
||||
*.xz
|
||||
logs/
|
||||
*.rdb
|
||||
dist/
|
||||
654
proj_level_reading/PROPOSAL/VKIST_MSK_PILOT_PROPOSAL.md
Normal file
654
proj_level_reading/PROPOSAL/VKIST_MSK_PILOT_PROPOSAL.md
Normal file
@@ -0,0 +1,654 @@
|
||||
# VKIST MSK Ultrasound Stack — Investment Proposal & System Specification
|
||||
|
||||
> **Audience:** CTO / Department Head / Executive Sponsor (technical-fluent, outcome-driven, risk-averse on healthcare investments)
|
||||
> **Author:** VKIST MSK Pilot Engineering Team
|
||||
> **Status:** Pilot proposal — active cycle June 2 – September 2, 2026 (strict 3-month window)
|
||||
> **Reading time:** Section 0 (Executive Summary) is a self-contained one-pager. Sections 1–8 provide the full technical and financial justification.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
0. [Executive Summary (one page)](#0-executive-summary-one-page)
|
||||
1. [Project Context — Who, What Pain, What Constraints](#1-project-context)
|
||||
2. [How the System Addresses the Problem](#2-how-the-system-addresses-the-problem)
|
||||
3. [Qualitative ROI — Behavioral & Workflow Change](#3-qualitative-roi--behavioral--workflow-change)
|
||||
4. [Quantitative ROI — Projected Metrics & Measurement Plan](#4-quantitative-roi--projected-metrics--measurement-plan)
|
||||
5. [Risk & Failure-Mode Handling — Why This Is Not Risky](#5-risk--failure-mode-handling--why-this-is-not-risky)
|
||||
6. [Compliance & Data Governance](#6-compliance--data-governance)
|
||||
7. [Roadmap & Milestones](#7-roadmap--milestones)
|
||||
8. [Investment Ask & Scope Boundaries](#8-investment-ask--scope-boundaries)
|
||||
|
||||
---
|
||||
|
||||
## 0. Executive Summary (one page)
|
||||
|
||||
**The problem, in one sentence.** Vietnamese public-hospital musculoskeletal (MSK) care runs on clinicians processing 100+ evaluations per shift with no time for objective double-checks, no shared visibility across the radiologist → surgeon → physiotherapist → patient chain, and patients so information-starved that they turn to dangerous folk remedies — while VKIST's proven computer-vision research models sit idle as static academic checkpoints.
|
||||
|
||||
**What we are building.** The **VKIST MSK Ultrasound Stack** is an **air-gapped-first, single-hospital AI platform** that turns knee ultrasound scans into automated, explainable **synovitis grading (0–3)** with Vietnamese-language reasoning, every clinical statement **cited back to Ministry of Health (MOH) guidelines**, and a **human clinician signature required** before anything is finalized. It migrates VKIST's existing DeepLabv3/UNet/ConvNeXt vision models from research into a referenceable, production-grade medical software ecosystem.
|
||||
|
||||
**Why it is a sound investment (the four pillars):**
|
||||
|
||||
| Pillar | What it means for the leader |
|
||||
| :--- | :--- |
|
||||
| **Zero-friction adoption** | The AI is an *invisible background validation layer* — 0 extra clicks to see explanations (NFR-12/13). Clinicians keep their workflow; the system removes liability risk instead of adding work. |
|
||||
| **Non-negotiable safety** | Human-in-the-Loop signature gate (NFR-19), 100% MOH-cited explanations (NFR-18), immutable audit log (NFR-17), and a 4-quadrant safety framework that explicitly defends against automation bias *and* AI hallucination. |
|
||||
| **Regulatory sovereignty** | Runs inside the hospital LAN. Decree 13/2023 PII protection + Circular 46/2018 EMR compliance are built into the architecture, not bolted on. No patient data leaves the hospital in production. |
|
||||
| **Capital efficiency** | Built on proven open-source (K3s, Triton, Postgres/pgvector, Keycloak, MinIO) and existing VKIST models. No SaaS lock-in, no per-seat licensing, delivered in a fixed 3-month pilot. |
|
||||
|
||||
**The ask.** Sponsorship of a fixed 3-month pilot on a single hospital's on-premise infrastructure (one Dell PowerEdge class node + one GPU node), a small cross-functional team, and a *governed, time-boxed* cloud allowance for the proof-of-concept LLM tiers (NFR-16a, retired at sign-off). See [Section 8](#8-investment-ask--scope-boundaries).
|
||||
|
||||
**The expected return.** A validated, referenceable clinical AI product that (projected) removes tedious measurement work from overloaded specialists, gives physiotherapists diagnostic visibility they have never had, converts patient anxiety into institutional trust, and establishes VKIST's operational credibility in the local healthcare community — with every claim instrumented and measurable during the pilot itself (see [Section 4](#4-quantitative-roi--projected-metrics--measurement-plan)).
|
||||
|
||||
> **Bottom line for the decision-maker:** This is a low-blast-radius, standards-compliant, single-site pilot that converts already-funded research into a safe, auditable clinical tool. The downside is bounded (one hospital, 3 months, open-source stack); the upside is a defensible product line and clinical-community trust.
|
||||
|
||||
---
|
||||
|
||||
## 1. Project Context
|
||||
|
||||
### 1.1 Who the System Is Designed For
|
||||
|
||||
The platform coordinates a **four-persona clinical care chain**. The pilot (Sprint 1–2, FR-25) is centered on the **Diagnostic Radiologist (UP5)** as the primary human actor, with the other three personas as governance/future-sprint stakeholders whose needs shape the non-functional requirements.
|
||||
|
||||
| Persona | Who they are | Environment | Core responsibility | What the system must respect |
|
||||
| :--- | :--- | :--- | :--- | :--- |
|
||||
| **UP5 — Diagnostic Radiologist** | Specialist MDs, age 28–55, from digitally native attendings to department heads | Low-light reading rooms; massive daily DICOM backlogs; asynchronous, detached from patients | Translate images into definitive reports that gate all downstream care | *Invisible workflow preservation* — no extra clicks, no DICOM render lag, no uncompensated patient chat |
|
||||
| **UP7 — Rheumatologist / Orthopedic Surgeon** | Treatment architects, age 30–60; seniors control procurement | Split across OR, chaotic outpatient clinics (>100 patients/shift), inpatient wards | Synthesize fragmented data into a treatment plan; bear full legal/clinical liability | *Clinical force multiplier* — rapid dashboards, automated patient-education visuals, strictly gated communication |
|
||||
| **UP6 — Physical Therapist** | Young (36–40% aged 20–29), highly stratified (≈54% vocational tier), digitally literate | Cramped (<20 m²), high-noise public units; 11–20+ patients/shift; shared/scarce workstations | Execute physician scripts into kinetic rehab programs | *Digital exoskeleton* — gel-proof knuckle-tap UI, zero-GPU mobile, localized language, objective progress data |
|
||||
| **UP8 — MSK Patient & Family Caregiver** | Elderly patients (45–80+) via younger caregiver proxies (20–45) | Home, pharmacy, crowded waiting rooms; saturated with health misinformation | Manage chronic pain; adhere to protocols; make family-centric decisions | *Empathy + accessibility* — jargon-free, high-contrast, large-font, dual-profile family access on legacy phones |
|
||||
|
||||
*Source: [User_Research_Result.md](../Requirement_Analysis/USER_RESEARCH/User_Research_Result.md), [CONTEXT_VISION_SCOPE.md §3](../PLAN/CONTEXT_VISION_SCOPE.md).*
|
||||
|
||||
### 1.2 The Pain Points That Drive the Need
|
||||
|
||||
These are not hypothetical — they are drawn from ethnographic user research (professional forums, patient forums, and published studies).
|
||||
|
||||
- **Clinical exhaustion & volume:** Central referral hospitals (Bach Mai, Viet Duc) regularly exceed **100 evaluations per shift**, leaving no time for tedious measurements or objective double-checks. This creates the risk of *automation-bias blind-concurrence* — a clinician rubber-stamping an AI estimate under time pressure.
|
||||
- **The physiotherapist information bottleneck:** PTs receive only brief text prescriptions — no DICOM, no scan findings — forcing **semi-blind therapeutic execution** against estimated surface anatomy.
|
||||
- **The physiotherapist body & literacy crisis:** **76.4%** WMSD prevalence (career-shortening physical strain); **84%** report a foreign-language barrier (only 25% adequate English); 96%/93%/86% rely on personal experience/peer chat/old textbooks rather than formal evidence.
|
||||
- **Dangerous folk interventions:** Information-starved patients turn to leaf-wrapping or violent manual adjustment, causing tissue infection or permanent anatomical failure — because no one translated their scan into understandable terms.
|
||||
- **Stranded research value:** VKIST's CV models (DeepLabv3/UNet/ConvNeXt/MedViT) are proven in the lab but have no production pathway, so they generate zero operational or reputational value.
|
||||
|
||||
### 1.3 Constraints the System Must Meet
|
||||
|
||||
**Functional Requirements (pilot scope — FR-25 pipeline):**
|
||||
|
||||
| FR | Requirement |
|
||||
| :--- | :--- |
|
||||
| **FR-23/24/25** | Segment knee joint structures → measure synovium thickness (mm) → grade synovitis **0–3** |
|
||||
| **FR-25 / Grad-CAM** | Spatial activation overlay on the primary viewport with **zero extra clicks** |
|
||||
| **FR-25 / Safety loop** | Conversational Circuit Breaker + Socratic dialogue before finalization; BERT drift monitor; RAG-Referee arbitration |
|
||||
| **FR-25 / Reporting** | Circular 46/2018 compliant PDF report generation |
|
||||
| **FR-25 / Finalization** | Cryptographically signed, HITL-gated electronic record → EMR |
|
||||
|
||||
**Non-Functional Requirements (the "why it won't fail" constraints):**
|
||||
|
||||
| NFR | Constraint | Why the leader cares |
|
||||
| :--- | :--- | :--- |
|
||||
| **NFR-4** | ≤ 150 MB idle app bundle | Runs on cheap, legacy hospital/consumer hardware |
|
||||
| **NFR-5** | ≤ 1.5 s core inference (on-prem) | No workflow slowdown |
|
||||
| **NFR-7** | ≤ 200 ms token streaming (TTFT) | Feels instantaneous to the clinician |
|
||||
| **NFR-9** | ≥ 99.9% availability (≤ 45 s downtime/day) | Clinically dependable |
|
||||
| **NFR-10** | 100% of patient-facing LLM text passes safety verification | No unsafe generative output reaches a human |
|
||||
| **NFR-11** | Onboarding ≤ 45 minutes | Cheap to roll out to busy staff |
|
||||
| **NFR-12/13** | 0 extra clicks for explainability; native spatial activation maps | Anti-black-box; builds clinician trust |
|
||||
| **NFR-14** | No client-side GPU/accelerator required (Android 10+, 3 GB RAM) | Works on the fleet that actually exists |
|
||||
| **NFR-15** | Circular 46/2018 EMR compliance | Legally deployable |
|
||||
| **NFR-16** | Air-gapped primary (no PHI over public internet in production) | Data sovereignty (Decree 13) |
|
||||
| **NFR-17** | Immutable audit log | Accountability & liability defense |
|
||||
| **NFR-18** | 100% LLM clinical text cites MOH protocol via RAG | Every claim is traceable & defensible |
|
||||
| **NFR-19** | HITL digital signature before FINALIZED/ARCHIVED | A licensed human always owns the decision |
|
||||
|
||||
*Source: [SOLUTION_ARCHITECTURE_SPEC.md §2](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md), [SOFTWARE_ARCHITECTURE_SPEC.md §2](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md).*
|
||||
|
||||
### 1.4 System Context — C4 Level 1 (PlantUML)
|
||||
|
||||
The context diagram summarizes *who* and *what* the system boundary talks to. The pilot activates UP5 (and governance actors UP1/UP4); UP6/UP7/UP8 are shown as future-sprint boundaries.
|
||||
|
||||
```plantuml
|
||||
@startuml VKIST_MSK_System_Context
|
||||
!include <C4/C4_Context>
|
||||
|
||||
title System Context Diagram — VKIST MSK Ultrasound Stack (Pilot: FR-25 Synovitis Grading)
|
||||
|
||||
' === PRIMARY HUMAN ACTOR (in pilot scope) ===
|
||||
Person(radiologist, "Diagnostic Radiologist (UP5)", "Loads knee ultrasound, reviews AI grading (0-3), confirms/overrides, engages Socratic circuit breaker, finalizes & signs the report, reads GradCAM + RAG evidence")
|
||||
Person(admin, "System Administrator", "Deploys/monitors on-prem K3s, model updates, observability, NGINX failover")
|
||||
|
||||
' === GOVERNANCE / NFR-ALIGNMENT ACTORS ===
|
||||
Person_Ext(senior_expert, "Healthcare Senior Expert (UP1)", "Clinical protocol validation, model-threshold approval, pilot sponsor")
|
||||
Person_Ext(support_staff, "Support Staff (UP4)", "Patient registration, case-queue management")
|
||||
|
||||
' === FUTURE-SPRINT PERSONAS (out of pilot scope) ===
|
||||
Person_Ext(therapist, "Physical Therapist (UP6)", "Future: read-only diagnostic visibility, kinetic overlays, progress tracking")
|
||||
Person_Ext(surgeon, "Surgeon / Rheumatologist (UP7)", "Future: aggregated dashboards, treatment planning")
|
||||
Person_Ext(patient, "Patient & Caregiver (UP8)", "Future: patient portal, plain-language education, self-monitoring")
|
||||
|
||||
' === THE SYSTEM ===
|
||||
System(mpps, "VKIST MSK Ultrasound Stack (FR-25)", "Air-gapped-first: knee ultrasound AI (angle -> inflammation -> segmentation), synovitis grading 0-3 with GradCAM, 4-quadrant clinical safety, MOH-cited Vietnamese explanations, HITL-signed Circular 46 reports")
|
||||
|
||||
' === EXTERNAL SYSTEMS ===
|
||||
System_Ext(pacs, "Hospital PACS / Ultrasound Machine", "DICOM source (C-MOVE + direct capture)")
|
||||
System_Ext(emr, "Hospital EMR / HIS", "Signed report storage, prescription/lab sync (HL7/FHIR)")
|
||||
System_Ext(triton, "Triton Inference Server", "GPU node: 3-step ML pipeline + RAG embeddings (gRPC)")
|
||||
System_Ext(knowledge, "Clinical Knowledge Systems", "ladybugDB ontology (SNOMED-CT), pgvector MOH guideline index, Gemma/MedGemma LLM tiers")
|
||||
|
||||
' === RELATIONSHIPS ===
|
||||
Rel(radiologist, mpps, "Reviews grade, overrides, finalizes & signs, reads explanations", "HTTPS")
|
||||
Rel(admin, mpps, "Deploys, monitors, updates models", "HTTPS / SSH")
|
||||
Rel(senior_expert, mpps, "Validates protocols, approves thresholds", "HTTPS (governance)")
|
||||
Rel(support_staff, mpps, "Registration, case queue", "HTTPS")
|
||||
Rel(therapist, mpps, "Future sprint scope", "—")
|
||||
Rel(surgeon, mpps, "Future sprint scope", "—")
|
||||
Rel(patient, mpps, "Future sprint scope", "—")
|
||||
|
||||
Rel(mpps, pacs, "Imports DICOM", "DICOM / C-MOVE")
|
||||
Rel(mpps, emr, "Pushes signed reports + audit records", "HL7/FHIR")
|
||||
Rel(mpps, triton, "ML inference offload", "gRPC 8001")
|
||||
Rel(mpps, knowledge, "RAG retrieval, ontology traversal, LLM generation", "SQL (pgvector) + in-process")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
*Adapted from [SOLUTION_ARCHITECTURE_SPEC.md §8.1](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md).*
|
||||
|
||||
### 1.5 Applicability — Use-Case View (PlantUML)
|
||||
|
||||
The pilot's functional scope is organized around a **4-Quadrant Human-AI Interaction Framework** that deliberately defends against both *automation bias* (blindly trusting AI) and *clinician subservience* (AI overriding a correct human). This is the applicability model — what the radiologist can actually *do* with the system.
|
||||
|
||||
```plantuml
|
||||
@startuml VKIST_MSK_UseCases
|
||||
left to right direction
|
||||
skinparam packageStyle rectangle
|
||||
|
||||
actor "Diagnostic Radiologist (UP5)" as Rad
|
||||
actor "MSK Vision Grader" as Grader <<system>>
|
||||
actor "LLM Explainer" as LLM <<system>>
|
||||
actor "RAG-Referee Arbitrator" as Referee <<system>>
|
||||
actor "Hospital EMR" as EMR <<system>>
|
||||
|
||||
rectangle "VKIST MSK Workspace (FR-25)" {
|
||||
usecase "UC-48376\nLoad Patient Scan Session" as U1
|
||||
usecase "UC-47988\nReview Suggested Synovitis Grade (0-3)" as U2
|
||||
usecase "UC-92006\nFinalize & Sign Electronic Record" as U3
|
||||
|
||||
rectangle "Q1 — True Agreement (AI correct / Doctor correct)" {
|
||||
usecase "UC-25776\nGradCAM + CoT Explanation" as Q1a
|
||||
usecase "UC-02423\nLog High-Trust Concur Block" as Q1b
|
||||
}
|
||||
rectangle "Q2 — Automation Override (AI correct / Doctor oversight)" {
|
||||
usecase "UC-22159\nTrigger Circuit Breaker" as Q2a
|
||||
usecase "UC-55146\nSocratic Reasoning Dialogue" as Q2b
|
||||
usecase "UC-74821\nBERT Context-Drift Monitor" as Q2c
|
||||
usecase "UC-65473\nArbitrate via RAG-Referee" as Q2d
|
||||
}
|
||||
rectangle "Q3 — Clinician Subservience (AI hallucinates / Doctor correct)" {
|
||||
usecase "UC-25637\nExpose Pixel-Level Activation" as Q3a
|
||||
usecase "UC-60739\nIsolate Visual Noise/Artifacts" as Q3b
|
||||
usecase "UC-62864\nCommit Validated Ground-Truth" as Q3c
|
||||
}
|
||||
rectangle "Q4 — Double-Blind Failure (AI faulty / edge case)" {
|
||||
usecase "UC-35956\nActivate Clinical Investigation Mode" as Q4a
|
||||
usecase "UC-47796\nStructured Morphology Annotation" as Q4b
|
||||
usecase "UC-01580\nSerialize to Telemetry Queue" as Q4c
|
||||
}
|
||||
}
|
||||
|
||||
Rad --> U1
|
||||
Rad --> U2
|
||||
Rad --> U3
|
||||
Rad --> Q2b : Argue observations
|
||||
Rad --> Q3b : Tag artifacts
|
||||
Rad --> Q4b : Document manual findings
|
||||
|
||||
Grader --> U1 : Feeds vision tensors + scores
|
||||
Grader --> U2 : Generates prediction
|
||||
LLM --> Q1a : Generates clinical CoT
|
||||
LLM --> Q2b : Generates Socratic checks
|
||||
Referee --> Q2d : Retrieves MOH guidelines
|
||||
U3 --> EMR : Syncs signed report
|
||||
Q1b --> EMR : Stores consensus log
|
||||
Q4c --> EMR : Routes anomaly to engineering telemetry
|
||||
|
||||
U1 ..> Q1a : <<include>>
|
||||
Q1a ..> Q1b : <<include>>
|
||||
U2 <.. Q2a : <<extend>> (friction detected)
|
||||
Q2a ..> Q2b : <<include>>
|
||||
Q2a ..> Q2c : <<include>>
|
||||
Q2c ..> Q2d : <<extend>> (impasse/drift)
|
||||
U2 <.. Q3a : <<extend>> (clinician contests score)
|
||||
Q3a ..> Q3b : <<include>>
|
||||
Q3b ..> Q3c : <<include>>
|
||||
U2 <.. Q4a : <<extend>> (low confidence + empty RAG)
|
||||
Q4a ..> Q4b : <<include>>
|
||||
Q4b ..> Q4c : <<include>>
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
*Adapted from [full_usecase_planuml.md](../../workspace/sprint_1_2/Design_Material/FR_25_UC_DESIGN/Use_Case/full_usecase_planuml.md) and [CONTEXT_FR_25_UC_ELIT.md](../Requirement_Analysis/UC_Design/FR_25_UC_DESIGN/CONTEXT_FR_25_UC_ELIT.md).*
|
||||
|
||||
---
|
||||
|
||||
## 2. How the System Addresses the Problem
|
||||
|
||||
### 2.1 Technology Choices — Problem → Solution → Why Optimal → Limitation
|
||||
|
||||
Each technology is selected against a specific user problem or NFR, not for novelty. The table states the honest limitation of each choice so the leader sees a balanced picture.
|
||||
|
||||
| Technology | User problem / NFR it solves | Why it is the optimal choice | Limitation / trade-off |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **React PWA + LiteRT (MediaPipe) + WebLLM edge** | NFR-14 (no client GPU), NFR-4 (150 MB), zero-friction on legacy phones | Installable, offline-capable, runs a local angle pre-classifier + edge LLM with no server round-trip; degrades gracefully on weak hardware | Browser memory ceilings; edge LLM is small (GemmaE2B-Q4 ~1.3 GB) so deep reasoning must escalate |
|
||||
| **FastAPI edge-inference-svc (Python 3.12)** | FR-25 pipeline orchestration, Decree 13 middleware, SSE streaming | Async, lightweight, tight control over PII redaction + audit at the boundary; in-house domain logic | Python GIL for CPU-bound work — mitigated by offloading inference to Triton |
|
||||
| **Triton Inference Server (ONNX/TensorRT)** | NFR-5 (≤1.5 s inference), reuse of VKIST CV models | Industry-standard, GPU-efficient, self-hosted (no SaaS); serves the 3-step ensemble + embeddings | Requires a GPU node (capex); one more component to operate |
|
||||
| **Postgres + pgvector (HNSW)** | NFR-18 MOH-cited RAG, transactional consistency of clinical records | One store for records *and* vectors (~15K guideline vectors) → ~5–20 ms retrieval, no separate vector DB to run | Not optimal beyond ~100K vectors / 500 QPS (documented Phase-2 path to Qdrant) |
|
||||
| **RAG-Referee (BERT classifier)** | NFR-10/18 — reject unsupported LLM claims | Validates attribution + logical cohesion of every explanation before it reaches a human | Adds latency to the cloud tier; thresholds need clinical tuning |
|
||||
| **3-Tier LLM (Edge Gemma → Gemini → MedGemma)** | Vietnamese explanations, deep clinical reasoning, cost control | Cheapest-capable-tier routing: local first, cloud only when needed, <20% MedGemma usage cap | Cloud tiers are PoC-only (NFR-16a) and must be redacted — retired at sign-off |
|
||||
| **Keycloak (OIDC/RBAC) on-prem** | NFR-16 sovereignty, role separation (PT read-only) | No SaaS identity; realm-scoped RBAC inside the LAN | Operational overhead vs. managed Auth0/Okta (accepted for sovereignty) |
|
||||
| **K3s on Dell PowerEdge** | NFR-16 air-gap, NFR-9 availability | Lightweight Kubernetes for a single hospital; rolling updates, self-healing pods; NGINX+Keepalived VIP failover (≤2 s) | Single-site scale; not a multi-region HA design (by design for the pilot) |
|
||||
| **MinIO + Redis (5 scoped types)** | Object/blob store + session/cache | S3-compatible, self-hosted, no cloud egress; Redis scoped to prevent sprawl | Self-managed durability (mitigated by backup cron to MinIO) |
|
||||
|
||||
**Honest, system-level limitations (stated up front for the leader):**
|
||||
- **Single-hospital scale by design.** The pilot targets one site; multi-site federation is a documented Phase-2 evolution, not a pilot promise.
|
||||
- **Cloud LLM tiers are proof-of-concept only** under NFR-16a, gated by redaction + consent + audit, and **formally retired at PoC sign-off** — production reverts to fully air-gapped NFR-16.
|
||||
- **Edge model is intentionally small** — deep reasoning escalates through the tier chain, which is why the cost guard (<20% MedGemma) and RAG-Referee exist.
|
||||
|
||||
*Source: [SOFTWARE_ARCHITECTURE_SPEC.md §7 Build-vs-Buy](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md), [SOLUTION_ARCHITECTURE_SPEC.md §5](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md).*
|
||||
|
||||
### 2.2 Deployment Environment — C4 Container Diagram (PlantUML)
|
||||
|
||||
The system runs entirely inside the hospital LAN. The only external touchpoints are the governed, PoC-scoped cloud fallbacks (NFR-16a).
|
||||
|
||||
```plantuml
|
||||
@startuml VKIST_MSK_Containers
|
||||
!include <C4/C4_Container>
|
||||
|
||||
title C4 Container Diagram — VKIST MSK Ultrasound Stack
|
||||
|
||||
Person(radiologist, "Diagnostic Radiologist (UP5)", "Loads DICOM, reviews grading, finalizes reports")
|
||||
Person(admin, "System Administrator", "K3s ops, model updates, observability, failover")
|
||||
Person_Ext(senior_expert, "Healthcare Senior Expert (UP1)", "Protocol validation, threshold approval")
|
||||
Person_Ext(support_staff, "Support Staff (UP4)", "Registration, case queue")
|
||||
|
||||
System_Boundary(hospital_lan, "Hospital LAN (Air-Gapped, <=10 Mbps)") {
|
||||
Container(pwa, "React PWA Frontend", "React 18, TS, Zustand, LiteRT, MediaPipe, Dexie.js", "View-angle validation (WASM); DICOM preview; Grad-CAM overlay; Decree 13 scrubber; encrypted IndexedDB sessions")
|
||||
Container(nginx, "Active-Passive Gateway", "NGINX + Keepalived", "SSL termination, VIP load balancing, <=2 s failover")
|
||||
|
||||
System_Boundary(k3s_cluster, "K3s Orchestration Cluster") {
|
||||
Container(edge_inference, "Edge Inference Service", "FastAPI (Python 3.12)", "DICOM ingest, 3-step ML orchestration, Grad-CAM, report assembly, SSE streaming")
|
||||
Container(cloud_llm_gateway, "Cloud LLM Gateway", "FastAPI (Python 3.12)", "Routes to Gemini (orchestration) / MedGemma (clinical); NFR-16a redaction, consent, audit, cost guard")
|
||||
Container(rag_svc, "RAG & Knowledge Service", "FastAPI (asyncpg)", "pgvector top-k, ladybugDB ontology, RAG-Referee validation, mandatory RAG pre-processing")
|
||||
Container(audit_svc, "Audit & EMR Service", "Node.js / FastAPI Worker", "Immutable append-only audit; HL7/FHIR EMR push with outbox retry")
|
||||
Container(api_gw, "API Gateway", "Envoy", "TLS, rate limiting, routing, OIDC pass-through")
|
||||
Container(auth_svc, "Identity & Access", "Keycloak", "OIDC, RBAC, realm vkist-msk")
|
||||
Container(obs_stack, "Observability Stack", "Prometheus + Grafana", "Metrics, dashboards, alerting")
|
||||
}
|
||||
|
||||
System_Boundary(db_vm, "Database VM") {
|
||||
ContainerDb(postgres, "PostgreSQL + pgvector", "PostgreSQL 16 + pgvector", "EMR records, spatial markers, MOH HNSW index, embeddings, audit ledger")
|
||||
ContainerDb(redis, "Redis Cache", "Redis 7 (AOF+RDB)", "Sessions, guideline chunks, DICOM metadata, consult_mode state, rate-limit")
|
||||
ContainerDb(minio, "MinIO Object Store", "MinIO S3-compatible", "DICOM payloads, overlays, Grad-CAM heatmaps, report PDFs, model staging")
|
||||
}
|
||||
}
|
||||
|
||||
System_Ext(triton, "Triton Inference Server", "NVIDIA Triton (ONNX/TensorRT)", "3-step ML pipeline + EmbeddingGemma RAG embeddings")
|
||||
System_Ext(emr, "Hospital EMR / HIS", "HL7/FHIR", "Signed report storage, prescription sync")
|
||||
System_Ext(pacs, "PACS / Ultrasound Device", "DICOM / C-MOVE", "Image capture & retrieval")
|
||||
System_Ext(vertex_ai, "GCP Vertex AI (Gemini)", "REST", "PoC-only NFR-16a: orchestration, translation; redacted payloads only")
|
||||
System_Ext(modal_medgemma, "Modal MedGemma", "T4 GPU", "PoC-only NFR-16a: clinical deep-reasoning; redacted + RAG-Referee validated")
|
||||
|
||||
Rel(radiologist, pwa, "Loads scan, reviews grade, finalizes, reads explanations", "HTTPS 443")
|
||||
Rel(admin, nginx, "Deploys, monitors, configures", "HTTPS / SSH")
|
||||
Rel(senior_expert, pwa, "Validates protocols, approves thresholds", "HTTPS 443")
|
||||
Rel(support_staff, pwa, "Registration, case queue", "HTTPS 443")
|
||||
|
||||
Rel(pwa, nginx, "API requests + pre-validated DICOM", "HTTPS 443")
|
||||
Rel(nginx, api_gw, "Routes upstream", "HTTP 8000")
|
||||
Rel(api_gw, edge_inference, "Forwards /api/*", "HTTP 8000")
|
||||
Rel(api_gw, rag_svc, "Forwards /rag/*", "HTTP 8001")
|
||||
Rel(api_gw, auth_svc, "Validates OIDC tokens", "HTTP 8080")
|
||||
|
||||
Rel(edge_inference, triton, "3-step inference + embeddings", "gRPC 8001")
|
||||
Rel(edge_inference, postgres, "Reads guidelines; writes cases/embeddings/audit", "SQL 5432")
|
||||
Rel(edge_inference, redis, "Session, rate-limit, consult_mode", "TCP 6379")
|
||||
Rel(edge_inference, minio, "DICOM, overlays, reports", "S3 API")
|
||||
Rel(edge_inference, cloud_llm_gateway, "Routes cloud consult", "HTTP 8000")
|
||||
|
||||
Rel(cloud_llm_gateway, vertex_ai, "Gemini proxy (NFR-16a redacted)", "HTTPS 443")
|
||||
Rel(cloud_llm_gateway, modal_medgemma, "MedGemma proxy (NFR-16a redacted)", "HTTPS 443")
|
||||
Rel(cloud_llm_gateway, postgres, "Audit log append", "SQL 5432")
|
||||
|
||||
Rel(rag_svc, postgres, "pgvector HNSW queries", "SQL 5432")
|
||||
Rel(rag_svc, redis, "Guideline cache, pub/sub invalidation", "TCP 6379")
|
||||
|
||||
Rel(audit_svc, postgres, "Appends immutable audit events", "SQL 5432")
|
||||
Rel(audit_svc, emr, "Finalized report push", "HL7/FHIR")
|
||||
|
||||
Rel(pwa, pacs, "Direct DICOM capture / C-MOVE", "DICOM")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
*Reused from [SOFTWARE_ARCHITECTURE_SPEC.md §4.2](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md).*
|
||||
|
||||
### 2.3 Deployment Topology — C4 Deployment Diagram (PlantUML)
|
||||
|
||||
Physical placement: everything on hospital hardware; the only external node is the governed NFR-16a fallback.
|
||||
|
||||
```plantuml
|
||||
@startuml VKIST_MSK_Deployment
|
||||
!include <C4/C4_Deployment>
|
||||
LAYOUT_LEFT_RIGHT()
|
||||
|
||||
Deployment_Node(hw, "Dell PowerEdge (Hospital)") {
|
||||
Deployment_Node(k3s, "K3s Cluster") {
|
||||
Deployment_Node(pod_edge, "Pod: edge-inference-svc") {
|
||||
Container(ci, "FastAPI", "Python 3.12, Uvicorn")
|
||||
}
|
||||
Deployment_Node(pod_rag, "Pod: rag-svc") {
|
||||
Container(cr, "FastAPI", "Python 3.12, asyncpg")
|
||||
}
|
||||
Deployment_Node(pod_audit, "Pod: audit-svc") {
|
||||
Container(ca, "Node.js", "WAL writer")
|
||||
}
|
||||
Deployment_Node(pod_gw, "Pod: api-gateway") {
|
||||
Container(cg, "Envoy", "TLS, rate limit")
|
||||
}
|
||||
Deployment_Node(pod_auth, "Pod: auth-svc") {
|
||||
Container(ck, "Keycloak", "OIDC, RBAC")
|
||||
}
|
||||
Deployment_Node(pod_obs, "Pod: observability") {
|
||||
Container(cp, "Prometheus")
|
||||
Container(cf, "Grafana")
|
||||
}
|
||||
}
|
||||
Deployment_Node(db_vm, "DB VM") {
|
||||
ContainerDb(pg, "PostgreSQL + pgvector HNSW")
|
||||
ContainerDb(rd, "Redis")
|
||||
ContainerDb(mn, "MinIO")
|
||||
}
|
||||
}
|
||||
Deployment_Node(triton_node, "GPU Node (Triton)", "NVIDIA A10/T4, 24 GB VRAM") {
|
||||
Container(tc, "Triton Inference Server", "ONNX/TensorRT, gRPC 8001")
|
||||
}
|
||||
Deployment_Node(ext, "External (NFR-16a governed, PoC only)") {
|
||||
ContainerDb(s3v, "GCP Vertex AI / Modal MedGemma / CDN fallback")
|
||||
}
|
||||
|
||||
Rel(ci, pg, "SQL", "TCP 5432")
|
||||
Rel(ci, rd, "Cache", "TCP 6379")
|
||||
Rel(ci, mn, "Blobs", "S3 API")
|
||||
Rel(ci, tc, "Inference", "gRPC 8001")
|
||||
Rel(ci, ck, "Auth", "OIDC")
|
||||
Rel(ci, ca, "Audit", "HTTP")
|
||||
Rel(k3s, db_vm, "SQL / Cache / Blobs", "TCP")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
*Reused from [SOFTWARE_ARCHITECTURE_SPEC.md §4.4](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md).*
|
||||
|
||||
### 2.4 Components & Interactions — C4 Component Diagram (PlantUML)
|
||||
|
||||
Inside the core `edge-inference-svc`, each component maps to a specific functional requirement. The **Inference Router** implements the tiered LLM escalation; the **Circuit Breaker** guarantees the system fails safely to MOH templates.
|
||||
|
||||
```plantuml
|
||||
@startuml VKIST_MSK_Component_EdgeInference
|
||||
!include <C4/C4_Component>
|
||||
|
||||
Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") {
|
||||
Component(api, "API Controller", "REST + SSE", "/api/analyze, cloud-orchestrate redirect")
|
||||
Component(stream, "SSE Token Streamer", "StreamingResponse", "200 ms TTFT, 30 s heartbeat")
|
||||
Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub")
|
||||
Component(router, "Inference Router", "consult_mode state", "Tier selection: Edge Gemma -> Gemini -> MedGemma -> Templates")
|
||||
Component(breaker, "Circuit Breaker", "pybreaker", "Wraps Triton + EMR + Cloud LLM; fail-open to templates")
|
||||
Component(pipeline, "ML Pipeline", "gRPC client", "Angle -> Inflammation -> Segmentation -> Measurement")
|
||||
Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay -> base64 PNG")
|
||||
Component(report, "Report Builder", "WeasyPrint", "Circular 46 PDF; HITL signature gate")
|
||||
Component(rag, "RAG Service", "pgvector SQL", "Top-5 MOH chunks; mandatory pre-generation; NFR-18")
|
||||
Component(referee, "RAG-Referee", "BERT classifier", "Reject LLM text if citation confidence < threshold")
|
||||
Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; clean residual PII")
|
||||
Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override")
|
||||
}
|
||||
|
||||
Rel(api, stream, "delegates", "SSE")
|
||||
Rel(api, preproc, "validates image", "sync")
|
||||
Rel(api, router, "selects tier", "sync")
|
||||
Rel(router, breaker, "guards calls", "sync")
|
||||
Rel(router, pipeline, "invokes", "gRPC")
|
||||
Rel(router, stream, "fallback text", "SSE")
|
||||
Rel(api, gradcam, "requests overlay", "sync")
|
||||
Rel(api, report, "generates PDF", "sync")
|
||||
Rel(api, rag, "queries MOH", "SQL")
|
||||
Rel(rag, referee, "validates citations", "sync")
|
||||
Rel(api, nlp, "scrubs output", "sync")
|
||||
Rel(api, audit, "writes event", "sync")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
The **Cloud LLM Gateway** enforces the NFR-16a governance boundary — nothing egresses without consent, redaction, RAG grounding, and an audit record.
|
||||
|
||||
```plantuml
|
||||
@startuml VKIST_MSK_Component_CloudGateway
|
||||
!include <C4/C4_Component>
|
||||
|
||||
Container_Boundary(cloud_gw, "cloud-llm-gateway (FastAPI)") {
|
||||
Component(capi, "Cloud API Controller", "REST + SSE", "/api/cloud-orchestrate, /api/cloud-consult")
|
||||
Component(gateway, "Cloud LLM Router", "task_type matcher", "Orchestration/translation -> Gemini; clinical -> MedGemma")
|
||||
Component(consent, "Consent Enforcer", "Redis + NFR-16a checklist", "Validates consent token + redaction manifest before egress")
|
||||
Component(caudit, "Cloud Audit Emitter", "PostgreSQL append-only", "egress_consent, egress_response, escalation events")
|
||||
Component(cost, "Cost Guard", "Redis + Prometheus", "Tracks MedGemma usage; alerts if >20% over 24h")
|
||||
Component(creferee, "RAG-Referee Trigger", "BERT classifier", "Mandatory validation of MedGemma output")
|
||||
Component(crag, "RAG Pre-processing", "pgvector SQL", "Mandatory top-k injection before generation")
|
||||
}
|
||||
|
||||
Rel(capi, consent, "verifies", "sync")
|
||||
Rel(capi, gateway, "routes", "sync")
|
||||
Rel(gateway, crag, "injects chunks", "sync")
|
||||
Rel(gateway, creferee, "validates output", "sync")
|
||||
Rel(gateway, caudit, "logs egress", "sync")
|
||||
Rel(gateway, cost, "increments counter", "sync")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
*Reused from [SOFTWARE_ARCHITECTURE_SPEC.md §4.3](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md).*
|
||||
|
||||
---
|
||||
|
||||
## 3. Qualitative ROI — Behavioral & Workflow Change
|
||||
|
||||
The core value thesis: the system does not add a tool to a workflow — it **enriches the existing workflow and reconnects a fragmented care chain**. The following before/after captures the behavioral change per persona.
|
||||
|
||||
| Persona | Before (current behavior) | After (with the system) | Behavioral shift |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **UP5 Radiologist** | Manual, repetitive measurements under fatigue; risk of missing subtle findings; isolated in a reading room; liability anxiety | AI pre-measures + grades; Grad-CAM shows *why* with 0 extra clicks; a Socratic circuit breaker catches rushed sign-offs; every report is MOH-cited and audit-backed | From lone, fatigued gatekeeper → to a *validated* decision-maker shielded from liability |
|
||||
| **UP7 Surgeon** | Synthesizes fragmented silos by hand; no time for patient education; fears uncompensated patient messaging | Aggregated view of signed findings; auto-generated patient-friendly visuals; strictly gated (no open chat) communication | From data-assembler → to *clinical force multiplier* with protected time |
|
||||
| **UP6 Physiotherapist** | Semi-blind execution from a one-line prescription; no scan visibility; language barrier; manual charting | Read-only diagnostic visibility down the chain; objective progress metrics; gel-proof, localized, zero-GPU UI | From guesswork executor → to *data-backed* practitioner with professional standing |
|
||||
| **UP8 Patient/Caregiver** | Anxiety + confusion → folk remedies; crowdsources raw scans on social media | Scan translated into plain-language, high-tech visual validation of the doctor's diagnosis | From anxious misinformation-seeker → to a *trusting, informed* participant |
|
||||
|
||||
**System-level qualitative returns:**
|
||||
- **Institutional trust & credibility:** A high-trust, objective, MOH-anchored tool displaces both dangerous folk interventions (patients) and unsecured external messaging (clinicians bypassing the system).
|
||||
- **Academic-to-production migration:** VKIST research models become a live, referenceable clinical asset — establishing the team's operational value in the local healthcare community.
|
||||
- **Care-chain cohesion:** The radiologist's sign-off *ripples* to the surgeon, PT, and patient — turning isolated actions into a connected clinical relationship (the "Synchronized Ritual" concept).
|
||||
|
||||
*Source: [Motivation.md](../DEMO_EXP/Motivation.md), [User_Research_Result.md](../Requirement_Analysis/USER_RESEARCH/User_Research_Result.md).*
|
||||
|
||||
---
|
||||
|
||||
## 4. Quantitative ROI — Projected Metrics & Measurement Plan
|
||||
|
||||
> **Important framing for the leader:** The pilot has not yet deployed (runs through Sep 2026), so the figures below are **modeled projections**, not measured outcomes. Each row states its **baseline source** and its **assumption**, and — critically — **how the pilot will actually measure it**. The value of the pilot is precisely to *convert these projections into evidence*.
|
||||
|
||||
### 4.1 Projected Before → After Metrics
|
||||
|
||||
| Metric | Baseline (before) | Target (after) | Assumption / basis | How the pilot measures it |
|
||||
| :--- | :--- | :--- | :--- | :--- |
|
||||
| **Tedious measurement time per scan** | Manual synovium measurement + grading, several minutes/scan under load | Automated pre-measurement; radiologist verifies | FR-25 automates FR-23/24 (segment + measure) | Time-on-task instrumentation (session telemetry, NFR audit timestamps) |
|
||||
| **Core inference latency** | N/A (no automated pipeline today) | ≤ 1.5 s (on-prem) | NFR-5 target on Triton GPU node | Prometheus p50/p99 latency dashboards |
|
||||
| **Explainability access cost** | Not available / manual literature lookup | **0 extra clicks** (Grad-CAM + citation inline) | NFR-12/13 design mandate | UI event audit (clicks-to-explanation = 0) |
|
||||
| **Clinical claim traceability** | Ad-hoc, uncited | **100%** of LLM clinical text cites MOH protocol | NFR-18 mandatory RAG | RAG-Referee pass rate; citation coverage log |
|
||||
| **Unsafe generative output reaching a human** | N/A | **0%** (100% pass safety verification) | NFR-10 3-tier guardrail | Guardrail rejection log; audit events |
|
||||
| **Onboarding time per staff member** | Ad-hoc, hours | ≤ 45 minutes | NFR-11 target | Timed onboarding sessions during rollout |
|
||||
| **System availability** | Dependent on manual workflow | ≥ 99.9% (≤ 45 s downtime/day) | NFR-9 target | Uptime monitoring (Prometheus/Grafana) |
|
||||
| **Cloud (MedGemma) usage ratio** | N/A | < 20% of consults | Cost-guard target (capital efficiency) | Cost Guard Redis counter + alert |
|
||||
| **PT diagnostic visibility** | ~0 (text prescription only) | Read-only access to signed findings | RBAC role (future sprint) | Access-log coverage per PT-linked case |
|
||||
| **Auditability of AI accept/override** | None | 100% immutable capture | NFR-17 | Immutable audit ledger completeness check |
|
||||
|
||||
### 4.2 Behavioral-Change Indicators (Adoption Proxies)
|
||||
|
||||
Beyond raw performance, the pilot measures whether behavior actually changed — the true ROI signal for a leader:
|
||||
|
||||
- **Retention / anti-bypass:** % of target clinicians using the structured workspace instead of unsecured external messaging (a stated success criterion).
|
||||
- **Override rate & ground-truth capture:** frequency of clinician overrides (Q3) feeding the retraining corpus — proves the human-AI loop works both ways.
|
||||
- **Circuit-breaker engagement:** how often the Socratic breaker prevents a rushed sign-off (a direct automation-bias safety signal).
|
||||
- **Compliance events:** zero unencrypted PHI leak events (Decree 13) — a binary go/no-go for production.
|
||||
|
||||
*Baselines from [User_Research_Result.md](../Requirement_Analysis/USER_RESEARCH/User_Research_Result.md); targets from NFRs in [SOLUTION_ARCHITECTURE_SPEC.md §2.2](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md); success criteria from [CONTEXT_VISION_SCOPE.md §6](../PLAN/CONTEXT_VISION_SCOPE.md).*
|
||||
|
||||
---
|
||||
|
||||
## 5. Risk & Failure-Mode Handling — Why This Is Not Risky
|
||||
|
||||
The single most important message for an investment decision: **this system is engineered around the assumption that either the AI *or* the human can be wrong.** It handles all four combinations explicitly via the **4-Quadrant Human-AI Interaction Framework**.
|
||||
|
||||
```plantuml
|
||||
@startuml VKIST_MSK_Risk_Quadrants
|
||||
skinparam packageStyle rectangle
|
||||
title 4-Quadrant Human-AI Safety Framework (FR-25)
|
||||
|
||||
rectangle "Q1 — True Agreement\n(AI correct / Doctor correct)" as Q1
|
||||
rectangle "Q2 — Automation Override\n(AI correct / Doctor oversight)" as Q2
|
||||
rectangle "Q3 — Clinician Subservience\n(AI hallucinates / Doctor correct)" as Q3
|
||||
rectangle "Q4 — Double-Blind Failure\n(AI faulty / edge case)" as Q4
|
||||
|
||||
Q1 --> Q1 : GradCAM + CoT -> High-Trust Concur log
|
||||
Q2 --> Q2 : Circuit Breaker -> Socratic -> BERT drift -> RAG-Referee
|
||||
Q3 --> Q3 : Expose activations -> isolate artifacts -> commit ground-truth
|
||||
Q4 --> Q4 : Investigation mode -> manual annotation -> telemetry queue
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
| Risk quadrant | The danger | How the system mitigates it |
|
||||
| :--- | :--- | :--- |
|
||||
| **Q1 — True Agreement** | Even correct consensus needs a trace for liability | Grad-CAM + Chain-of-Thought explanation, then an immutable High-Trust Concur log block |
|
||||
| **Q2 — Automation Bias** | Overloaded doctor rubber-stamps AI | Conversational Circuit Breaker intercepts rushed finalization → Socratic dialogue → BERT drift monitor → RAG-Referee arbitration against MOH guidelines |
|
||||
| **Q3 — AI Hallucination** | AI is wrong, doctor is right, but AI "authority" pressures the human | Expose pixel-level activation logic; let the clinician mask artifacts and **commit the corrected ground-truth** (which also feeds retraining) |
|
||||
| **Q4 — Double-Blind Edge Case** | Both AI confidence and guideline coverage are low | Auto-switch to strict manual **Clinical Investigation Mode**; force structured morphology annotation; route the anomaly to an engineering telemetry queue (bypassing the EMR) |
|
||||
|
||||
**Cross-cutting safety guarantees:**
|
||||
- **HITL gate (NFR-19):** No AI output becomes FINALIZED/ARCHIVED without a licensed clinician's cryptographic signature. *A human always owns the decision.*
|
||||
- **Fail-safe by design:** The Circuit Breaker fails **open to deterministic MOH templates** — if Triton, EMR, or a cloud LLM is down, the system degrades to a safe rule-based floor, not to an error.
|
||||
- **Immutable audit (NFR-17):** Every accept/override/tier-transition/consent event is append-only and tamper-proof — a complete liability defense record.
|
||||
- **RAG-Referee (NFR-18):** No uncited clinical claim reaches a human.
|
||||
|
||||
*Source: [full_usecase_planuml.md](../../workspace/sprint_1_2/Design_Material/FR_25_UC_DESIGN/Use_Case/full_usecase_planuml.md), and the Q1–Q4 scenario docs under [FR_25_UC_DESIGN/](../Requirement_Analysis/UC_Design/FR_25_UC_DESIGN/).*
|
||||
|
||||
---
|
||||
|
||||
## 6. Compliance & Data Governance
|
||||
|
||||
Healthcare compliance is treated as a first-class architectural constraint, not an afterthought.
|
||||
|
||||
| Regulation | Requirement | How the architecture satisfies it |
|
||||
| :--- | :--- | :--- |
|
||||
| **Decree 13/2023/ND-CP** (personal health data) | Sensitive PII must be protected; data sovereignty | Client-side WebCrypto scrubbing + FastAPI Presidio redaction middleware; role-hash tokens `[BN:7f3a2c]`; all production data resides in-hospital / sovereign VN data centers |
|
||||
| **Circular 46/2018/TT-BYT** (EMR) | National EMR compliance for stored records | Circular-46-compliant PDF reports; HL7/FHIR EMR push; NFR-15 (not relaxed even under NFR-16a) |
|
||||
| **Air-gap (NFR-16, permanent)** | No diagnostic/identifiable data over public internet in production | Everything runs on the hospital LAN; K3s network policies isolate workloads; local-only container registries |
|
||||
| **Governed cloud PoC (NFR-16a, temporary)** | Cloud LLM only as an emergency/PoC fallback | Invoked only when edge LLM unavailable **and** user consents; mandatory pre-egress redaction + audit commit; CSEK keys, 30-day lifecycle, 1-year auto-delete; **retired at PoC sign-off** |
|
||||
|
||||
**The compliance headline for a leader:** in production, **no patient data leaves the hospital**, every clinical statement is cited, every action is audit-logged, and a human signature is legally required to finalize anything.
|
||||
|
||||
*Source: [SOLUTION_ARCHITECTURE_SPEC.md §2.2, §5.8](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md), [CONTEXT_VISION_SCOPE.md §4](../PLAN/CONTEXT_VISION_SCOPE.md).*
|
||||
|
||||
---
|
||||
|
||||
## 7. Roadmap & Milestones
|
||||
|
||||
A strict, time-boxed 3-month cycle (June 2 – September 2, 2026), executed as a hybrid of waterfall macro-phases and agile sprints. This bounds the investment and provides clear go/no-go checkpoints - However it can be tentative to change based on the actual resource of development team.
|
||||
|
||||
```mermaid
|
||||
gantt
|
||||
title VKIST MSK Pilot - 3-Month Delivery Roadmap
|
||||
dateFormat YYYY-MM-DD
|
||||
axisFormat %b %d
|
||||
|
||||
section Foundation
|
||||
Sprint 0 Pitch and Architecture Spike :done, sp0, 2026-06-02, 2026-06-12
|
||||
Win project pitch :milestone, m1, 2026-06-09
|
||||
Sprint 1 Fast PoC Baseline :active, sp1, 2026-06-15, 2026-06-26
|
||||
|
||||
section Core Build
|
||||
Sprint 2 Multi-Modal and NLP Integration :sp2, 2026-06-29, 2026-07-10
|
||||
Sprint 3 Collaborative Workspace :sp3, 2026-07-13, 2026-07-24
|
||||
|
||||
section Patient and Hardening
|
||||
Sprint 4 Patient-Facing PWA :sp4, 2026-07-27, 2026-08-07
|
||||
Sprint 5 Feedback Pipeline and Hardening :sp5, 2026-08-10, 2026-08-21
|
||||
Sprint 6 Beta Release and Evolution :sp6, 2026-08-24, 2026-09-02
|
||||
Pilot cycle complete :milestone, m2, 2026-09-02
|
||||
```
|
||||
|
||||
| Sprint | Window | Focus & key deliverable |
|
||||
| :--- | :--- | :--- |
|
||||
| **0** | Jun 02–12 | Pitch & architecture clearance; VKIST ML model audit; CI/CD baseline |
|
||||
| **1** | Jun 15–26 | Fast end-to-end PoC: FastAPI pipeline + inference mask preview |
|
||||
| **2** | Jun 29–Jul 10 | Multi-modal NLP + Decree 13 client-side scrubbing |
|
||||
| **3** | Jul 13–24 | Collaborative canvas; RBAC (PT read-only); usability testing |
|
||||
| **4** | Jul 27–Aug 07 | Installable PWA; zero-GPU fallback for legacy phones |
|
||||
| **5** | Aug 10–21 | Feedback instrumentation; performance/caching; cleanup |
|
||||
| **6** | Aug 24–Sep 02 | Beta launch to MSK cohort; telemetry analysis; Phase-2 backlog |
|
||||
|
||||
**Definition of success (pilot go/no-go):** zero workflow friction added; deterministic cross-device stability (incl. CPU-bound rendering); zero Decree 13 leak events; sustained clinician retention over external messaging; successful academic-to-production model migration.
|
||||
|
||||
*Source: [CONTEXT_VISION_SCOPE.md §5–6](../PLAN/CONTEXT_VISION_SCOPE.md).*
|
||||
|
||||
---
|
||||
|
||||
## 8. Investment Ask & Scope Boundaries
|
||||
|
||||
### 8.1 The Resource Envelope (qualitative)
|
||||
|
||||
| Category | Pilot ask | Rationale |
|
||||
| :--- | :--- | :--- |
|
||||
| **On-prem compute** | 1 × Dell PowerEdge-class node (K3s + DB VM) + 1 × GPU node (A10/T4, 24 GB VRAM) for Triton | Single-site air-gapped deployment; capex-only, no recurring SaaS |
|
||||
| **Team** | Small cross-functional squad (backend/ML, frontend, infra/DevOps) + clinical champion (UP1/UP5) for validation | Matches the 7-sprint scope; clinical sponsor is essential for governance & buy-in |
|
||||
| **Software** | Open-source stack (K3s, Triton, Postgres/pgvector, Redis, MinIO, Keycloak, Prometheus/Grafana) | Zero licensing cost; no vendor lock-in |
|
||||
| **Governed cloud (PoC only)** | Time-boxed GCP Vertex AI (Gemini) + Modal (MedGemma) budget, capped at <20% consult usage | NFR-16a fallback; **retired at sign-off** — not a recurring production cost |
|
||||
| **Data** | Access to de-identified MSK ultrasound + MOH guideline corpus | Needed to tune grading thresholds and RAG index |
|
||||
|
||||
### 8.2 In Scope vs. Out of Scope (Pilot)
|
||||
|
||||
| In scope (pilot) | Out of scope (Phase 2+) |
|
||||
| :--- | :--- |
|
||||
| FR-25 knee synovitis grading (UP5 primary) | Multi-hospital federation / multi-region HA |
|
||||
| Grad-CAM explainability, Socratic safety loop, RAG-Referee | Full UP7 surgeon planning & UP8 patient portal at production scale |
|
||||
| Decree 13 + Circular 46 compliance, HITL sign-off, audit log | Predictive analytics (cartilage degeneration, hardware failure forecasting) |
|
||||
| Single-hospital air-gapped deployment | Qdrant hot-tier vector cache (only if >100K vectors / >500 QPS) |
|
||||
| PoC cloud LLM tiers (NFR-16a, retired at sign-off) | Permanent cloud dependency (explicitly prohibited by NFR-16) |
|
||||
|
||||
### 8.3 Go / No-Go Decision Criteria
|
||||
|
||||
At the end of the 3-month cycle, the leader decides on Phase 2 based on objective evidence:
|
||||
|
||||
1. **Safety:** zero Decree 13 leak events; 100% HITL-gated finalizations; 100% MOH-cited clinical text.
|
||||
2. **Performance:** inference ≤ 1.5 s; availability ≥ 99.9%; onboarding ≤ 45 min.
|
||||
3. **Adoption:** measurable clinician retention within the structured workspace vs. external messaging.
|
||||
4. **Strategic:** VKIST models successfully running in a referenceable production form.
|
||||
|
||||
> **The investment is bounded and reversible:** one hospital, one quarter, open-source stack, no production cloud dependency. If the go/no-go criteria are not met, the blast radius is a single controlled pilot — not an enterprise commitment.
|
||||
|
||||
---
|
||||
|
||||
## Appendix — Source Documents
|
||||
|
||||
- Vision, personas, constraints, milestones: [CONTEXT_VISION_SCOPE.md](../PLAN/CONTEXT_VISION_SCOPE.md)
|
||||
- Deep personas + quantitative baselines: [User_Research_Result.md](../Requirement_Analysis/USER_RESEARCH/User_Research_Result.md)
|
||||
- Architecture, FR/NFR, C4 diagrams, build-vs-buy: [SOFTWARE_ARCHITECTURE_SPEC.md](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md), [SOLUTION_ARCHITECTURE_SPEC.md](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md)
|
||||
- Use cases + 4-quadrant framework: [full_usecase_planuml.md](../../workspace/sprint_1_2/Design_Material/FR_25_UC_DESIGN/Use_Case/full_usecase_planuml.md), [CONTEXT_FR_25_UC_ELIT.md](../Requirement_Analysis/UC_Design/FR_25_UC_DESIGN/CONTEXT_FR_25_UC_ELIT.md)
|
||||
- Risk scenarios: [FR_25_UC_DESIGN/](../Requirement_Analysis/UC_Design/FR_25_UC_DESIGN/) (Q1–Q4 scenario docs)
|
||||
- Impact narrative: [Motivation.md](../DEMO_EXP/Motivation.md)
|
||||
|
||||
*End of proposal.*
|
||||
@@ -31,7 +31,7 @@ os.environ.setdefault("TRITON_ENDPOINT", DEFAULT_TRITON_ENDPOINT)
|
||||
import uvicorn
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
import asyncio
|
||||
from backend.routers import cv_inference
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -44,10 +44,7 @@ async def lifespan(app: FastAPI):
|
||||
logger.info("Starting CV inference service on Triton: %s", os.getenv("TRITON_ENDPOINT"))
|
||||
from backend.services.triton_warmup import warmup_triton_models
|
||||
|
||||
try:
|
||||
await warmup_triton_models()
|
||||
except Exception as exc:
|
||||
logger.warning("Triton warmup failed — first clinical request may be slow: %s", exc)
|
||||
warmup_task = asyncio.create_task(warmup_triton_models())
|
||||
yield
|
||||
logger.info("Shutting down CV inference service")
|
||||
|
||||
|
||||
@@ -3,16 +3,34 @@ import json
|
||||
from typing import Any
|
||||
import numpy as np
|
||||
import requests
|
||||
from requests.adapters import HTTPAdapter
|
||||
from urllib3.util.retry import Retry
|
||||
|
||||
|
||||
class TritonAdapter:
|
||||
def __init__(self, endpoint_url: str, timeout: float = 60.0):
|
||||
self.endpoint_url = endpoint_url.rstrip("/")
|
||||
self.timeout = timeout
|
||||
self._session = self._build_session()
|
||||
|
||||
@staticmethod
|
||||
def _build_session() -> requests.Session:
|
||||
session = requests.Session()
|
||||
retry = Retry(
|
||||
total=0,
|
||||
connect=0,
|
||||
read=0,
|
||||
redirect=0,
|
||||
status=0,
|
||||
raise_on_status=False,
|
||||
)
|
||||
adapter = HTTPAdapter(max_retries=retry, pool_connections=20, pool_maxsize=50)
|
||||
session.mount("http://", adapter)
|
||||
session.mount("https://", adapter)
|
||||
return session
|
||||
|
||||
async def close(self):
|
||||
pass
|
||||
await asyncio.to_thread(self._session.close)
|
||||
|
||||
|
||||
async def infer(
|
||||
@@ -61,7 +79,7 @@ class TritonAdapter:
|
||||
}
|
||||
|
||||
url = f"{self.endpoint_url}/v2/models/{model_name}/infer"
|
||||
response = requests.post(url, data=body, headers=headers, timeout=self.timeout)
|
||||
response = self._session.post(url, data=body, headers=headers, timeout=self.timeout)
|
||||
response.raise_for_status()
|
||||
return self._parse_binary_response(response.headers, response.content)
|
||||
|
||||
@@ -91,7 +109,7 @@ class TritonAdapter:
|
||||
|
||||
def _model_ready_sync(self, model_name: str) -> bool:
|
||||
url = f"{self.endpoint_url}/v2/models/{model_name}"
|
||||
response = requests.get(url, timeout=self.timeout)
|
||||
response = self._session.get(url, timeout=self.timeout)
|
||||
if response.status_code == 404:
|
||||
return False
|
||||
response.raise_for_status()
|
||||
@@ -113,7 +131,7 @@ class TritonAdapter:
|
||||
url = f"{self.endpoint_url}/v2/repository/index"
|
||||
|
||||
# 2. Change requests.get to requests.post with an empty json payload {}
|
||||
response = requests.post(url, json={}, timeout=self.timeout)
|
||||
response = self._session.post(url, json={}, timeout=self.timeout)
|
||||
response.raise_for_status()
|
||||
|
||||
data = response.json()
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
import asyncio
|
||||
import base64
|
||||
import io
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from backend.implementation.postprocessing.calibration import calibration_config_from_params
|
||||
from backend.services.cv_inference_service import CvInferenceOptions, run_single
|
||||
from backend.services.celery_app import celery_app
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _decode_base64_to_pil(b64_str: str) -> Image.Image:
|
||||
raw = base64.b64decode(b64_str)
|
||||
return Image.open(io.BytesIO(raw)).convert("RGB")
|
||||
|
||||
|
||||
@celery_app.task(bind=True, name="cv_inference.run_chunk", max_retries=2, default_retry_delay=10)
|
||||
def run_cv_chunk(self, chunk_payload: dict) -> list[dict]:
|
||||
images_b64 = chunk_payload.get("images_b64", [])
|
||||
frame_ids = chunk_payload.get("frame_ids", [])
|
||||
calibration_params = chunk_payload.get("calibration", {})
|
||||
model_versions = chunk_payload.get("model_versions")
|
||||
|
||||
if not images_b64:
|
||||
return []
|
||||
|
||||
images = [_decode_base64_to_pil(b64) for b64 in images_b64]
|
||||
calibration = calibration_config_from_params(calibration_params)
|
||||
options = CvInferenceOptions(
|
||||
calibration=calibration,
|
||||
model_versions=model_versions,
|
||||
use_cache=False,
|
||||
)
|
||||
|
||||
async def _run():
|
||||
return await asyncio.gather(*[
|
||||
run_single(img, frame_id=fid, options=options)
|
||||
for img, fid in zip(images, frame_ids)
|
||||
])
|
||||
|
||||
try:
|
||||
results = asyncio.run(_run())
|
||||
return results
|
||||
except Exception as exc:
|
||||
logger.exception("Celery chunk task failed: %s", exc)
|
||||
raise self.retry(exc=exc, countdown=10, max_retries=2)
|
||||
@@ -0,0 +1,29 @@
|
||||
import logging
|
||||
import sys
|
||||
from logging.handlers import RotatingFileHandler
|
||||
from pathlib import Path
|
||||
|
||||
LOG_DIR = Path("logs")
|
||||
LOG_DIR.mkdir(exist_ok=True)
|
||||
LOG_FORMAT = "%(asctime)s | %(levelname)-8s | %(name)s | %(message)s"
|
||||
LOG_DATE_FORMAT = "%Y-%m-%d %H:%M:%S"
|
||||
def setup_logging():
|
||||
root = logging.getLogger()
|
||||
root.setLevel(logging.DEBUG)
|
||||
root.handlers.clear()
|
||||
formatter = logging.Formatter(LOG_FORMAT, datefmt=LOG_DATE_FORMAT)
|
||||
console = logging.StreamHandler(sys.stdout)
|
||||
console.setLevel(logging.INFO)
|
||||
console.setFormatter(formatter)
|
||||
root.addHandler(console)
|
||||
file_handler = RotatingFileHandler(
|
||||
LOG_DIR / "app.log",
|
||||
maxBytes=10 * 1024 * 1024,
|
||||
backupCount=5,
|
||||
encoding="utf-8",
|
||||
)
|
||||
file_handler.setLevel(logging.DEBUG)
|
||||
file_handler.setFormatter(formatter)
|
||||
root.addHandler(file_handler)
|
||||
logging.getLogger("httpx").setLevel(logging.WARNING)
|
||||
logging.getLogger("uvicorn.access").setLevel(logging.INFO)
|
||||
@@ -18,6 +18,10 @@ from backend.implementation.triton_batch import TRITON_MAX_BATCH_SIZE
|
||||
from backend.services import cv_result_cache
|
||||
from backend.services import triton_runtime_service as triton_runtime
|
||||
from backend.services.cv_inference_service import CvBatchResult, CvInferenceOptions, run_batch, run_single
|
||||
from backend.services import cv_celery_service
|
||||
from backend.logging.logging_config import setup_logging
|
||||
|
||||
setup_logging()
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -135,14 +139,15 @@ async def cv_inference_health():
|
||||
)
|
||||
|
||||
|
||||
@router.post("/analyze")
|
||||
@router.post("/analyze") # deprecated
|
||||
async def analyze_upload(
|
||||
image: UploadFile = File(...),
|
||||
calibration: str | None = Form(default=None),
|
||||
):
|
||||
"""Spec-compliant CV pipeline: CLAHE → angle → inflammation → conditional segmentation."""
|
||||
image_pil = await _load_upload_image(image)
|
||||
options = _build_options(_parse_calibration_form(calibration), use_cache=False)
|
||||
|
||||
options = _build_options(_parse_calibration_form(calibration), use_cache=True)
|
||||
|
||||
try:
|
||||
result = await run_single(image_pil, frame_id=None, options=options)
|
||||
@@ -168,11 +173,13 @@ async def analyze_batch_upload(
|
||||
calibration: str | None = Form(default=None),
|
||||
):
|
||||
"""Spec-compliant CV batch — one full pipeline per frame (angle-first, gated segmentation)."""
|
||||
logger.info("Starting analyze batch upload")
|
||||
if not images:
|
||||
raise HTTPException(status_code=400, detail="At least one image is required")
|
||||
|
||||
try:
|
||||
id_list = json.loads(frame_ids)
|
||||
# logger.info("Start to check the id_list")
|
||||
except json.JSONDecodeError as exc:
|
||||
raise HTTPException(status_code=400, detail="frame_ids must be a JSON array of strings") from exc
|
||||
|
||||
@@ -213,6 +220,68 @@ async def analyze_batch_upload(
|
||||
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.post("/analyze/batch/celery")
|
||||
async def analyze_batch_celery(
|
||||
images: list[UploadFile] = File(...),
|
||||
frame_ids: str = Form(...),
|
||||
calibration: str | None = Form(default=None),
|
||||
):
|
||||
"""Experiment: async chunk fan-out via Celery + Redis."""
|
||||
if not images:
|
||||
raise HTTPException(status_code=400, detail="At least one image is required")
|
||||
|
||||
try:
|
||||
id_list = json.loads(frame_ids)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise HTTPException(status_code=400, detail="frame_ids must be a JSON array of strings") from exc
|
||||
|
||||
if not isinstance(id_list, list) or not all(isinstance(x, str) for x in id_list):
|
||||
raise HTTPException(status_code=400, detail="frame_ids must be a JSON array of strings")
|
||||
if len(id_list) != len(images):
|
||||
raise HTTPException(status_code=400, detail="frame_ids length must match images count")
|
||||
|
||||
image_pils: list[Image.Image] = []
|
||||
for upload in images:
|
||||
image_pils.append(await _load_upload_image(upload))
|
||||
|
||||
options = _build_options(_parse_calibration_form(calibration))
|
||||
|
||||
try:
|
||||
job_id = cv_celery_service.submit_celery_batch(image_pils, id_list, options)
|
||||
return JSONResponse({
|
||||
"success": True,
|
||||
"job_id": job_id,
|
||||
"image_count": len(image_pils),
|
||||
"mode": "celery-chunk-fanout",
|
||||
"chunk_size": cv_celery_service.CELERY_CHUNK_SIZE,
|
||||
})
|
||||
except Exception as exc:
|
||||
logger.exception("Celery batch submit failed")
|
||||
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.get("/analyze/batch/celery/{job_id}")
|
||||
async def analyze_batch_celery_result(job_id: str):
|
||||
"""Poll result for a Celery chunk-fan-out batch job."""
|
||||
try:
|
||||
result = await cv_celery_service.get_celery_batch_result(job_id)
|
||||
status = result.get("status")
|
||||
if status == "pending":
|
||||
return JSONResponse(
|
||||
status_code=202,
|
||||
content=result,
|
||||
)
|
||||
if status == "unknown":
|
||||
return JSONResponse(
|
||||
status_code=404,
|
||||
content=result,
|
||||
)
|
||||
return JSONResponse(result)
|
||||
except Exception as exc:
|
||||
logger.exception("Celery batch result fetch failed")
|
||||
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.post("/segment")
|
||||
@router.post("/segment/batch")
|
||||
@router.post("/angle")
|
||||
|
||||
27
workspace/sprint_1_2/CODEBASE/backend/routers/run_cv_inference.sh
Executable file
27
workspace/sprint_1_2/CODEBASE/backend/routers/run_cv_inference.sh
Executable file
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Launch the standalone CV inference FastAPI server (Modal Triton).
|
||||
#
|
||||
# Usage:
|
||||
# ./backend/run_cv_inference.sh
|
||||
#
|
||||
# Works from any directory: it resolves the CODEBASE root relative to this
|
||||
# script, sets PYTHONPATH so `import backend...` resolves, then runs the
|
||||
# server as a module.
|
||||
#
|
||||
# Override defaults via env vars, e.g.:
|
||||
# CV_INFERENCE_PORT=8080 ./backend/run_cv_inference.sh
|
||||
#
|
||||
set -euo pipefail
|
||||
|
||||
# CODEBASE root = grandparent dir of this script's directory (script lives in backend/routers/).
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
CODEBASE_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
|
||||
|
||||
cd "${CODEBASE_ROOT}"
|
||||
|
||||
export PYTHONPATH="${CODEBASE_ROOT}:${PYTHONPATH:-}"
|
||||
|
||||
# exec python -m backend.cv_inference_server
|
||||
|
||||
exec uvicorn backend.cv_inference_server:app --host 0.0.0.0 --port ${CV_INFERENCE_PORT:-8001}
|
||||
22
workspace/sprint_1_2/CODEBASE/backend/services/celery_app.py
Normal file
22
workspace/sprint_1_2/CODEBASE/backend/services/celery_app.py
Normal file
@@ -0,0 +1,22 @@
|
||||
from celery import Celery
|
||||
|
||||
from backend.implementation import config
|
||||
|
||||
celery_app = Celery(
|
||||
"cv-inference",
|
||||
broker=f"redis://{config.REDIS_HOST}:{config.REDIS_PORT}/{config.REDIS_DB}",
|
||||
backend=f"redis://{config.REDIS_HOST}:{config.REDIS_PORT}/{config.REDIS_DB}",
|
||||
# Explicit include — autodiscover looks for tasks.py, not cv_tasks.py
|
||||
include=["backend.implementation.tasks.cv_tasks"],
|
||||
)
|
||||
|
||||
# Must match the @task(name=...) value, not the Python module path
|
||||
celery_app.conf.task_routes = {
|
||||
"cv_inference.run_chunk": {"queue": "cv-inference"},
|
||||
}
|
||||
|
||||
celery_app.conf.task_serializer = "json"
|
||||
celery_app.conf.result_serializer = "json"
|
||||
celery_app.conf.accept_content = ["json"]
|
||||
celery_app.conf.result_expires = 3600
|
||||
celery_app.conf.task_default_queue = "cv-inference"
|
||||
@@ -0,0 +1,130 @@
|
||||
import base64
|
||||
import dataclasses
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from backend.services.cv_inference_service import CvInferenceOptions, _encode_image_to_bytes
|
||||
from backend.implementation.tasks.cv_tasks import run_cv_chunk
|
||||
from backend.services.celery_app import celery_app
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
CELERY_CHUNK_SIZE = int(os.getenv("CELERY_CHUNK_SIZE", "4"))
|
||||
CELERY_BATCH_POLL_CACHE_TTL_MS = float(os.getenv("CELERY_BATCH_POLL_CACHE_TTL_MS", "2000"))
|
||||
|
||||
_batch_poll_cache: dict[str, tuple[float, dict[str, Any]]] = {}
|
||||
_batch_timings: dict[str, float] = {}
|
||||
|
||||
|
||||
def submit_celery_batch(images, frame_ids, options):
|
||||
if not images:
|
||||
raise ValueError("images must not be empty")
|
||||
if len(frame_ids) != len(images):
|
||||
raise ValueError("frame_ids length must match images length")
|
||||
|
||||
from celery import group
|
||||
|
||||
chunks = []
|
||||
for i in range(0, len(images), CELERY_CHUNK_SIZE):
|
||||
chunk_imgs = images[i : i + CELERY_CHUNK_SIZE]
|
||||
chunk_fids = frame_ids[i : i + CELERY_CHUNK_SIZE]
|
||||
b64_images = [
|
||||
base64.b64encode(_encode_image_to_bytes(img)).decode() for img in chunk_imgs
|
||||
]
|
||||
chunk_payload = {
|
||||
"images_b64": b64_images,
|
||||
"frame_ids": chunk_fids,
|
||||
"calibration": dataclasses.asdict(options.calibration) if options.calibration else {},
|
||||
"model_versions": options.model_versions,
|
||||
}
|
||||
chunks.append(chunk_payload)
|
||||
|
||||
job = group(run_cv_chunk.s(chunk) for chunk in chunks)
|
||||
result = job.apply_async()
|
||||
try:
|
||||
result.save()
|
||||
except Exception:
|
||||
logger.exception("Failed to persist Celery GroupResult for job %s", result.id)
|
||||
|
||||
_batch_timings[result.id] = time.monotonic()
|
||||
logger.info(
|
||||
"Celery batch submitted job_id=%s chunks=%d images=%d",
|
||||
result.id,
|
||||
len(chunks),
|
||||
len(images),
|
||||
)
|
||||
return result.id
|
||||
|
||||
|
||||
async def get_celery_batch_result(job_id):
|
||||
from celery.result import GroupResult
|
||||
|
||||
now = time.monotonic()
|
||||
cached = _batch_poll_cache.get(job_id)
|
||||
if cached and cached[0] > now:
|
||||
return cached[1]
|
||||
|
||||
result = GroupResult.restore(job_id, app=celery_app)
|
||||
if result is None:
|
||||
payload = {
|
||||
"status": "unknown",
|
||||
"detail": "Job ID not found in result backend",
|
||||
}
|
||||
_batch_poll_cache[job_id] = (now + CELERY_BATCH_POLL_CACHE_TTL_MS / 1000, payload)
|
||||
return payload
|
||||
|
||||
completed = result.completed_count()
|
||||
total = len(result.results)
|
||||
|
||||
if not result.ready():
|
||||
payload = {
|
||||
"status": "pending",
|
||||
"completed": completed,
|
||||
"total": total,
|
||||
"progress": round(completed / total, 2) if total > 0 else 0,
|
||||
}
|
||||
_batch_poll_cache[job_id] = (now + CELERY_BATCH_POLL_CACHE_TTL_MS / 1000, payload)
|
||||
return payload
|
||||
|
||||
if result.failed():
|
||||
errors = []
|
||||
for r in result.results:
|
||||
if r.failed():
|
||||
errors.append(str(r.result))
|
||||
payload = {
|
||||
"status": "failed",
|
||||
"errors": errors,
|
||||
}
|
||||
_batch_poll_cache[job_id] = (now + CELERY_BATCH_POLL_CACHE_TTL_MS / 1000, payload)
|
||||
_log_batch_timing(job_id, "failed")
|
||||
return payload
|
||||
|
||||
all_results = []
|
||||
for r in result.results:
|
||||
chunk_results = r.get()
|
||||
all_results.extend(chunk_results)
|
||||
|
||||
payload = {
|
||||
"status": "completed",
|
||||
"image_count": len(all_results),
|
||||
"results": all_results,
|
||||
}
|
||||
_batch_poll_cache[job_id] = (now + CELERY_BATCH_POLL_CACHE_TTL_MS / 1000, payload)
|
||||
_log_batch_timing(job_id, "completed")
|
||||
return payload
|
||||
|
||||
|
||||
def _log_batch_timing(job_id: str, status: str) -> None:
|
||||
start = _batch_timings.pop(job_id, None)
|
||||
if start is not None:
|
||||
duration_ms = (time.monotonic() - start) * 1000
|
||||
logger.info(
|
||||
"Celery batch %s job_id=%s duration_ms=%.0f",
|
||||
status,
|
||||
job_id,
|
||||
duration_ms,
|
||||
)
|
||||
else:
|
||||
logger.info("Celery batch %s job_id=%s (no start time recorded)", status, job_id)
|
||||
@@ -5,6 +5,7 @@ import asyncio
|
||||
import base64
|
||||
import io
|
||||
import logging
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
@@ -52,8 +53,6 @@ SEGMENT_CLASSES_POST = {
|
||||
6: "baker's cyst",
|
||||
}
|
||||
|
||||
_triton_pipeline_lock = asyncio.Lock()
|
||||
|
||||
|
||||
@dataclass
|
||||
class CvInferenceOptions:
|
||||
@@ -76,6 +75,12 @@ def _encode_image_to_data_url(image_pil: Image.Image) -> str:
|
||||
return f"data:image/png;base64,{encoded}"
|
||||
|
||||
|
||||
def _encode_image_to_bytes(image_pil: Image.Image) -> bytes:
|
||||
buffered = io.BytesIO()
|
||||
image_pil.save(buffered, format="PNG")
|
||||
return buffered.getvalue()
|
||||
|
||||
|
||||
def _build_inflammation_payload(logits_row: np.ndarray, config: CalibrationConfig) -> dict:
|
||||
interpreted = interpret_inflammation_logits(logits_row, config)
|
||||
return {
|
||||
@@ -147,6 +152,9 @@ def _build_segmentation_result(
|
||||
classes_detected = [name for name, mask in masks.items() if np.sum(mask) > 0 and name != "background"]
|
||||
color_legend = _build_color_legend(classes_detected, angle_type)
|
||||
|
||||
segmented_png_bytes = _encode_image_to_bytes(overlay)
|
||||
segmented_b64 = base64.b64encode(segmented_png_bytes).decode()
|
||||
|
||||
result: dict[str, Any] = {
|
||||
"success": True,
|
||||
"angle": angle_payload,
|
||||
@@ -161,7 +169,7 @@ def _build_segmentation_result(
|
||||
},
|
||||
"images": {
|
||||
"enhanced": enhanced_data_url,
|
||||
"segmented": _encode_image_to_data_url(overlay),
|
||||
"segmented": f"data:image/png;base64,{segmented_b64}",
|
||||
},
|
||||
"models_used": {
|
||||
"angle": angle_model,
|
||||
@@ -281,16 +289,27 @@ async def _run_batch_uncached(
|
||||
if len(frame_ids) != len(images):
|
||||
raise ValueError("frame_ids length must match images length")
|
||||
|
||||
async with _triton_pipeline_lock:
|
||||
results: list[dict[str, Any]] = []
|
||||
infer_modes: list[str] = []
|
||||
triton_call_count = 0
|
||||
for image_pil, fid in zip(images, frame_ids, strict=True):
|
||||
item, mode, calls = await _run_spec_cv_pipeline_single(
|
||||
concurrency = int(os.getenv("CV_BATCH_CONCURRENCY", "2"))
|
||||
semaphore = asyncio.Semaphore(concurrency)
|
||||
|
||||
async def process_one(image_pil: Image.Image, fid: str, index: int):
|
||||
async with semaphore:
|
||||
if index > 0:
|
||||
await asyncio.sleep(min(index * 0.15, 1.0))
|
||||
return await _run_spec_cv_pipeline_single(
|
||||
image_pil,
|
||||
frame_id=fid,
|
||||
options=options,
|
||||
)
|
||||
|
||||
outcomes = await asyncio.gather(
|
||||
*[process_one(img, fid, idx) for idx, (img, fid) in enumerate(zip(images, frame_ids))],
|
||||
)
|
||||
|
||||
results: list[dict[str, Any]] = []
|
||||
infer_modes: list[str] = []
|
||||
triton_call_count = 0
|
||||
for item, mode, calls in outcomes:
|
||||
results.append(item)
|
||||
infer_modes.append(mode)
|
||||
triton_call_count += calls
|
||||
|
||||
@@ -25,12 +25,11 @@ INPUT_NAME = "input_image"
|
||||
OUTPUT_NAME = "logits"
|
||||
|
||||
TRITON_INFER_TIMEOUT = float(os.getenv("TRITON_INFER_TIMEOUT", "120"))
|
||||
TRITON_INFER_RETRIES = int(os.getenv("TRITON_INFER_RETRIES", "5"))
|
||||
TRITON_RETRY_BASE_S = float(os.getenv("TRITON_RETRY_BASE_S", "4"))
|
||||
TRITON_INFER_RETRIES = int(os.getenv("TRITON_INFER_RETRIES", "2"))
|
||||
TRITON_RETRY_BASE_S = float(os.getenv("TRITON_RETRY_BASE_S", "2.0"))
|
||||
TRITON_USE_BATCH_INFER = os.getenv("TRITON_USE_BATCH_INFER", "true").lower()
|
||||
RETRYABLE_STATUS = {429, 502, 503, 504}
|
||||
|
||||
_triton_infer_lock = asyncio.Lock()
|
||||
_adapter: TritonAdapter | None = None
|
||||
_adapter_endpoint: str | None = None
|
||||
|
||||
@@ -43,6 +42,11 @@ def _get_adapter() -> TritonAdapter:
|
||||
global _adapter, _adapter_endpoint
|
||||
endpoint = get_triton_endpoint()
|
||||
if _adapter is None or _adapter_endpoint != endpoint:
|
||||
if _adapter is not None:
|
||||
try:
|
||||
asyncio.get_event_loop().run_until_complete(_adapter.close())
|
||||
except RuntimeError:
|
||||
pass
|
||||
_adapter = TritonAdapter(endpoint_url=endpoint, timeout=TRITON_INFER_TIMEOUT)
|
||||
_adapter_endpoint = endpoint
|
||||
return _adapter
|
||||
@@ -201,7 +205,7 @@ def _infer_angle_logits_chunk(images: list[Image.Image], model_name: str) -> tup
|
||||
if not _should_try_batched_infer(len(images)):
|
||||
return _infer_angle_logits_sequential(images, model_name), "sequential"
|
||||
try:
|
||||
return _infer_angle_logits_batch(images, model_name, max_retries=1), "batched"
|
||||
return _infer_angle_logits_batch(images, model_name), "batched"
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"Batched angle infer×%s failed (%s); falling back to sequential single-image calls",
|
||||
@@ -249,7 +253,7 @@ def _infer_inflammation_logits_chunk(
|
||||
if not _should_try_batched_infer(len(images)):
|
||||
return _infer_inflammation_logits_sequential(images, model_name), "sequential"
|
||||
try:
|
||||
return _infer_inflammation_logits_batch(images, model_name, max_retries=1), "batched"
|
||||
return _infer_inflammation_logits_batch(images, model_name), "batched"
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"Batched inflammation infer×%s failed (%s); falling back to sequential",
|
||||
@@ -297,7 +301,7 @@ def _infer_segmentation_logits_chunk(
|
||||
if not _should_try_batched_infer(len(images)):
|
||||
return _infer_segmentation_logits_sequential(images, model_name), "sequential"
|
||||
try:
|
||||
return _infer_segmentation_logits_batch(images, model_name, max_retries=1), "batched"
|
||||
return _infer_segmentation_logits_batch(images, model_name), "batched"
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"Batched segmentation infer×%s failed (%s); falling back to sequential",
|
||||
@@ -311,8 +315,6 @@ async def infer_angle_logits(
|
||||
images: list[Image.Image],
|
||||
model_name: str,
|
||||
) -> tuple[np.ndarray, Literal["batched", "sequential"], int]:
|
||||
"""Run angle classification under the global Triton lock. Returns (logits, mode, call_count)."""
|
||||
async with _triton_infer_lock:
|
||||
all_logits: list[np.ndarray] = []
|
||||
modes: list[str] = []
|
||||
call_count = 0
|
||||
@@ -332,7 +334,6 @@ async def infer_inflammation_logits(
|
||||
images: list[Image.Image],
|
||||
model_name: str,
|
||||
) -> tuple[np.ndarray, Literal["batched", "sequential"], int]:
|
||||
async with _triton_infer_lock:
|
||||
all_logits: list[np.ndarray] = []
|
||||
modes: list[str] = []
|
||||
call_count = 0
|
||||
@@ -352,7 +353,6 @@ async def infer_segmentation_logits(
|
||||
images: list[Image.Image],
|
||||
model_name: str,
|
||||
) -> tuple[np.ndarray, Literal["batched", "sequential"], int]:
|
||||
async with _triton_infer_lock:
|
||||
all_logits: list[np.ndarray] = []
|
||||
modes: list[str] = []
|
||||
call_count = 0
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Warm up Modal Triton on CV service startup — reduces cold-start 502s and first-frame latency."""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
|
||||
@@ -23,6 +24,20 @@ def _warmup_model_versions() -> dict[str, str]:
|
||||
return versions
|
||||
|
||||
|
||||
async def _warmup_one(
|
||||
name: str,
|
||||
coro,
|
||||
timeout: float,
|
||||
) -> None:
|
||||
try:
|
||||
await asyncio.wait_for(coro, timeout=timeout)
|
||||
logger.info("Triton warmup %s complete", name)
|
||||
except asyncio.TimeoutError:
|
||||
logger.warning("Triton warmup %s timed out after %.1fs", name, timeout)
|
||||
except Exception as exc:
|
||||
logger.warning("Triton warmup %s failed: %s", name, exc)
|
||||
|
||||
|
||||
async def warmup_triton_models() -> None:
|
||||
if os.getenv("CV_SKIP_TRITON_WARMUP", "").lower() in {"1", "true", "yes"}:
|
||||
logger.info("Triton warmup skipped (CV_SKIP_TRITON_WARMUP)")
|
||||
@@ -36,13 +51,29 @@ async def warmup_triton_models() -> None:
|
||||
img224 = Image.new("RGB", (224, 224), color=(128, 128, 128))
|
||||
img512 = Image.new("RGB", (512, 512), color=(128, 128, 128))
|
||||
|
||||
warmup_timeout = float(os.getenv("TRITON_WARMUP_TIMEOUT", "15"))
|
||||
|
||||
logger.info(
|
||||
"Warming up Triton (angle=%s, inflammation=%s, segmentation=%s)…",
|
||||
"Warming up Triton (angle=%s, inflammation=%s, segmentation=%s, timeout=%.1fs)…",
|
||||
angle_model,
|
||||
inflam_model,
|
||||
seg_model,
|
||||
warmup_timeout,
|
||||
)
|
||||
|
||||
await _warmup_one(
|
||||
"angle",
|
||||
triton_runtime.infer_angle_logits_single(img224, angle_model),
|
||||
warmup_timeout,
|
||||
)
|
||||
await _warmup_one(
|
||||
"inflammation",
|
||||
triton_runtime.infer_inflammation_logits_single(img224, inflam_model),
|
||||
warmup_timeout,
|
||||
)
|
||||
await _warmup_one(
|
||||
"segmentation",
|
||||
triton_runtime.infer_segmentation_logits_single(img512, seg_model),
|
||||
warmup_timeout,
|
||||
)
|
||||
await triton_runtime.infer_angle_logits_single(img224, angle_model)
|
||||
await triton_runtime.infer_inflammation_logits_single(img224, inflam_model)
|
||||
await triton_runtime.infer_segmentation_logits_single(img512, seg_model)
|
||||
logger.info("Triton warmup complete")
|
||||
|
||||
179
workspace/sprint_1_2/CODEBASE/backend/start_celery_workers.sh
Executable file
179
workspace/sprint_1_2/CODEBASE/backend/start_celery_workers.sh
Executable file
@@ -0,0 +1,179 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Start Redis + Celery worker for CV inference experiments.
|
||||
#
|
||||
# Usage:
|
||||
# ./backend/start_celery_workers.sh # start both
|
||||
# ./backend/start_celery_workers.sh stop # stop both
|
||||
# ./backend/start_celery_workers.sh restart # restart both
|
||||
# ./backend/start_celery_workers.sh status # show status
|
||||
#
|
||||
# Logs:
|
||||
# logs/redis.log
|
||||
# logs/celery.log
|
||||
#
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
CODEBASE_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
LOG_DIR="${CODEBASE_ROOT}/logs"
|
||||
REDIS_PID_FILE="${LOG_DIR}/redis.pid"
|
||||
CELERY_PID_FILE="${LOG_DIR}/celery.pid"
|
||||
|
||||
mkdir -p "${LOG_DIR}"
|
||||
|
||||
cd "${CODEBASE_ROOT}"
|
||||
export PYTHONPATH="${CODEBASE_ROOT}:${PYTHONPATH:-}"
|
||||
|
||||
REDIS_PORT="${REDIS_PORT:-6379}"
|
||||
REDIS_DB="${REDIS_DB:-0}"
|
||||
CELERY_CHUNK_SIZE="${CELERY_CHUNK_SIZE:-4}"
|
||||
export TRITON_ENDPOINT="${TRITON_ENDPOINT:-https://dtj-tran--triton-s3-service-unified-triton-server.modal.run}"
|
||||
|
||||
|
||||
is_redis_running() {
|
||||
if [ -f "${REDIS_PID_FILE}" ]; then
|
||||
local pid
|
||||
pid=$(cat "${REDIS_PID_FILE}")
|
||||
if kill -0 "${pid}" 2>/dev/null; then
|
||||
return 0
|
||||
else
|
||||
rm -f "${REDIS_PID_FILE}"
|
||||
fi
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
|
||||
start_redis() {
|
||||
if is_redis_running; then
|
||||
echo "Redis already running (PID $(cat ${REDIS_PID_FILE}))"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "Starting Redis on port ${REDIS_PORT}..."
|
||||
redis-server \
|
||||
--port "${REDIS_PORT}" \
|
||||
--daemonize yes \
|
||||
--pidfile "${REDIS_PID_FILE}" \
|
||||
--logfile "${LOG_DIR}/redis.log" \
|
||||
--save "" \
|
||||
--appendonly no
|
||||
|
||||
sleep 0.5
|
||||
|
||||
if is_redis_running; then
|
||||
echo "Redis started (PID $(cat ${REDIS_PID_FILE}))"
|
||||
else
|
||||
echo "Redis failed to start. Check ${LOG_DIR}/redis.log"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
start_celery() {
|
||||
if [ -f "${CELERY_PID_FILE}" ] && kill -0 "$(cat "${CELERY_PID_FILE}")" 2>/dev/null; then
|
||||
echo "Celery already running (PID $(cat ${CELERY_PID_FILE}))"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "Starting Celery worker..."
|
||||
nohup celery \
|
||||
-A backend.services.celery_app \
|
||||
worker \
|
||||
--loglevel=info \
|
||||
-Q cv-inference \
|
||||
--concurrency=2 \
|
||||
> "${LOG_DIR}/celery.log" 2>&1 &
|
||||
|
||||
echo $! > "${CELERY_PID_FILE}"
|
||||
|
||||
sleep 1
|
||||
|
||||
if kill -0 "$(cat "${CELERY_PID_FILE}")" 2>/dev/null; then
|
||||
echo "Celery started (PID $(cat ${CELERY_PID_FILE}))"
|
||||
echo " Logs: ${LOG_DIR}/celery.log"
|
||||
else
|
||||
echo "Celery failed to start. Check ${LOG_DIR}/celery.log"
|
||||
rm -f "${CELERY_PID_FILE}"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
stop_redis() {
|
||||
if is_redis_running; then
|
||||
local pid
|
||||
pid=$(cat "${REDIS_PID_FILE}")
|
||||
echo "Stopping Redis (PID ${pid})..."
|
||||
kill "${pid}" 2>/dev/null || true
|
||||
sleep 0.5
|
||||
rm -f "${REDIS_PID_FILE}"
|
||||
echo "Redis stopped"
|
||||
else
|
||||
echo "Redis not running"
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
stop_celery() {
|
||||
if [ -f "${CELERY_PID_FILE}" ] && kill -0 "$(cat "${CELERY_PID_FILE}")" 2>/dev/null; then
|
||||
local pid
|
||||
pid=$(cat "${CELERY_PID_FILE}")
|
||||
echo "Stopping Celery (PID ${pid})..."
|
||||
kill "${pid}" 2>/dev/null || true
|
||||
sleep 1
|
||||
# Force kill if still running
|
||||
if kill -0 "${pid}" 2>/dev/null; then
|
||||
kill -9 "${pid}" 2>/dev/null || true
|
||||
fi
|
||||
rm -f "${CELERY_PID_FILE}"
|
||||
echo "Celery stopped"
|
||||
else
|
||||
echo "Celery not running"
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
status() {
|
||||
echo "=== Worker Status ==="
|
||||
if is_redis_running; then
|
||||
echo "Redis: running (PID $(cat ${REDIS_PID_FILE}))"
|
||||
else
|
||||
echo "Redis: stopped"
|
||||
fi
|
||||
|
||||
if [ -f "${CELERY_PID_FILE}" ] && kill -0 "$(cat "${CELERY_PID_FILE}")" 2>/dev/null; then
|
||||
echo "Celery: running (PID $(cat ${CELERY_PID_FILE}))"
|
||||
else
|
||||
echo "Celery: stopped"
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
case "${1:-start}" in
|
||||
start)
|
||||
start_redis
|
||||
start_celery
|
||||
echo ""
|
||||
echo "Workers ready. Test with:"
|
||||
echo " curl http://localhost:8001/api/test/analyze/batch/celery"
|
||||
;;
|
||||
stop)
|
||||
stop_celery
|
||||
stop_redis
|
||||
;;
|
||||
restart)
|
||||
stop_celery
|
||||
stop_redis
|
||||
start_redis
|
||||
start_celery
|
||||
;;
|
||||
status)
|
||||
status
|
||||
;;
|
||||
*)
|
||||
echo "Usage: $0 {start|stop|restart|status}"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
34
workspace/sprint_1_2/CODEBASE/frontend/implementation/.gitignore
vendored
Normal file
34
workspace/sprint_1_2/CODEBASE/frontend/implementation/.gitignore
vendored
Normal file
@@ -0,0 +1,34 @@
|
||||
# Dependencies
|
||||
node_modules/
|
||||
|
||||
# Build output
|
||||
dist/
|
||||
build/
|
||||
|
||||
# Vite
|
||||
.vite/
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
npm-debug.log*
|
||||
|
||||
# Environment files (migrated to config/frontend.config.yaml)
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
|
||||
# Test coverage
|
||||
coverage/
|
||||
|
||||
# Misc
|
||||
*.local
|
||||
@@ -0,0 +1,13 @@
|
||||
# Frontend application configuration
|
||||
# This is the single source of truth for frontend feature flags and URLs.
|
||||
# Edit this file directly instead of using .env / .env.development.
|
||||
|
||||
VITE_USE_BACKEND_SEGMENTATION: "true"
|
||||
VITE_SEGMENT_API_BASE: ""
|
||||
VITE_USE_CV_CELERY: "false"
|
||||
VITE_API_BASE_URL: ""
|
||||
VITE_CLINICAL_CHAT_USE_LLM: "true"
|
||||
VITE_CLINICAL_CHAT_MOCK_TOOLS: "true"
|
||||
VITE_OLLAMA_CHAT_URL: "/api/ollama-chat/api/chat"
|
||||
VITE_OLLAMA_MODEL: "gemma4:e4b"
|
||||
VITE_MODAL_OLLAMA_TARGET: "https://dtj-tran--ollama-gemma4-e4b-ollamaserver-web.modal.run"
|
||||
@@ -27,7 +27,7 @@ export default function CalibrationControls({ config, onChange }: CalibrationCon
|
||||
return (
|
||||
<div className="cal-ctrl">
|
||||
<div className="cal-ctrl__header">
|
||||
<span className="cal-ctrl__title">Điều chỉnh nhiệt độ (T)</span>
|
||||
<span className="cal-ctrl__title">Điều chỉnh độ chắc chắn (T)</span>
|
||||
<span className="cal-ctrl__hint tnum">T = {config.temperature.toFixed(2)}</span>
|
||||
</div>
|
||||
<CalibrationMetricHelp layout="block" />
|
||||
|
||||
@@ -32,10 +32,17 @@ const MODEL_LOAD_COPY: Record<
|
||||
'installing-gemma': {
|
||||
title: 'Đang cài đặt Gemma 4 E2B về máy…',
|
||||
subtitle:
|
||||
'Mô hình trò chuyện chính (~2 GB). Nếu bị gián đoạn, lần mở sau sẽ tiếp tục từ chỗ đã tải.',
|
||||
'Mô hình trò chuyện chính (~1.87 GB). Nếu bị gián đoạn, lần mở sau sẽ tiếp tục từ chỗ đã tải.',
|
||||
ariaLabel: 'Đang cài đặt Gemma 4 E2B',
|
||||
composerPlaceholder: 'Đang cài đặt Gemma 4 E2B, vui lòng đợi…',
|
||||
},
|
||||
'resuming-gemma': {
|
||||
title: 'Đang tiếp tục tải Gemma 4 E2B…',
|
||||
subtitle:
|
||||
'Lần tải trước bị gián đoạn — đang tiếp tục từ chỗ đã tải, không tải lại từ đầu.',
|
||||
ariaLabel: 'Đang tiếp tục tải Gemma 4 E2B',
|
||||
composerPlaceholder: 'Đang tiếp tục tải Gemma 4 E2B, vui lòng đợi…',
|
||||
},
|
||||
// 'installing-qwen': { ... },
|
||||
'loading-gemma': {
|
||||
title: 'Đang nạp Gemma 4 E2B…',
|
||||
@@ -78,6 +85,7 @@ export default function ClinicalChatPanel({
|
||||
modelLoadPhase,
|
||||
modelLoadProgress,
|
||||
modelInstallTransferLabel,
|
||||
modelLoadStalled,
|
||||
modelLoadFading,
|
||||
sendMessage,
|
||||
stopGeneration,
|
||||
@@ -150,8 +158,17 @@ export default function ClinicalChatPanel({
|
||||
const showLoadBubble = isModelLoading && modelLoadPhase !== null;
|
||||
const loadCopy = modelLoadPhase ? MODEL_LOAD_COPY[modelLoadPhase] : null;
|
||||
const progressPercent = Math.min(100, Math.max(0, Math.round(modelLoadProgress)));
|
||||
const installProgressLabel =
|
||||
modelLoadPhase && isModelInstallPhase(modelLoadPhase) && modelInstallTransferLabel
|
||||
const isInstallStalled =
|
||||
modelLoadStalled && modelLoadPhase !== null && isModelInstallPhase(modelLoadPhase);
|
||||
const loadTitle = isInstallStalled ? 'Tải Gemma bị gián đoạn' : loadCopy?.title;
|
||||
const loadSubtitle = isInstallStalled
|
||||
? `Mất kết nối — hiện không tải được. Đang thử lại tự động${
|
||||
modelInstallTransferLabel ? ` · đã lưu ${modelInstallTransferLabel}` : ''
|
||||
}. Có thể tải lại trang để tiếp tục từ chỗ đã tải.`
|
||||
: loadCopy?.subtitle;
|
||||
const installProgressLabel = isInstallStalled
|
||||
? 'Gián đoạn'
|
||||
: modelLoadPhase && isModelInstallPhase(modelLoadPhase) && modelInstallTransferLabel
|
||||
? modelInstallTransferLabel
|
||||
: `${progressPercent}%`;
|
||||
const activeModeMeta = getInferenceModeMeta(inferenceMode);
|
||||
@@ -340,20 +357,37 @@ export default function ClinicalChatPanel({
|
||||
aria-hidden
|
||||
/>
|
||||
<div
|
||||
className={`clinical-chat__load-bubble clinical-chat__load-bubble--${modelLoadPhase} ${modelLoadFading ? 'clinical-chat__load-bubble--fade-out' : ''}`}
|
||||
className={`clinical-chat__load-bubble clinical-chat__load-bubble--${modelLoadPhase} ${isInstallStalled ? 'clinical-chat__load-bubble--stalled' : ''} ${modelLoadFading ? 'clinical-chat__load-bubble--fade-out' : ''}`}
|
||||
role="status"
|
||||
aria-live="polite"
|
||||
aria-label={
|
||||
modelLoadPhase && isModelInstallPhase(modelLoadPhase) && modelInstallTransferLabel
|
||||
isInstallStalled
|
||||
? `${loadCopy.ariaLabel} bị gián đoạn, đang thử lại`
|
||||
: modelLoadPhase && isModelInstallPhase(modelLoadPhase) && modelInstallTransferLabel
|
||||
? `${loadCopy.ariaLabel}, ${modelInstallTransferLabel}`
|
||||
: `${loadCopy.ariaLabel}, ${progressPercent} phần trăm`
|
||||
}
|
||||
>
|
||||
<div
|
||||
className={`clinical-chat__load-icon clinical-chat__load-icon--${modelLoadPhase}`}
|
||||
className={`clinical-chat__load-icon clinical-chat__load-icon--${modelLoadPhase} ${isInstallStalled ? 'clinical-chat__load-icon--stalled' : ''}`}
|
||||
aria-hidden
|
||||
>
|
||||
{modelLoadPhase && isModelInstallPhase(modelLoadPhase) ? (
|
||||
{isInstallStalled ? (
|
||||
<svg width="20" height="20" viewBox="0 0 24 24" fill="none">
|
||||
<path
|
||||
d="M12 8v5M12 16.5v.5"
|
||||
stroke="currentColor"
|
||||
strokeWidth="1.75"
|
||||
strokeLinecap="round"
|
||||
/>
|
||||
<path
|
||||
d="M10.3 3.9 2.4 18a2 2 0 0 0 1.7 3h15.8a2 2 0 0 0 1.7-3L13.7 3.9a2 2 0 0 0-3.4 0Z"
|
||||
stroke="currentColor"
|
||||
strokeWidth="1.6"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
</svg>
|
||||
) : modelLoadPhase && isModelInstallPhase(modelLoadPhase) ? (
|
||||
<svg width="20" height="20" viewBox="0 0 24 24" fill="none">
|
||||
<path
|
||||
d="M12 3v3M12 18v3M4.22 4.22l2.12 2.12M17.66 17.66l2.12 2.12M3 12h3M18 12h3M4.22 19.78l2.12-2.12M17.66 6.34l2.12-2.12"
|
||||
@@ -373,16 +407,16 @@ export default function ClinicalChatPanel({
|
||||
</svg>
|
||||
)}
|
||||
</div>
|
||||
<p className="clinical-chat__load-title">{loadCopy.title}</p>
|
||||
<p className="clinical-chat__load-subtitle">{loadCopy.subtitle}</p>
|
||||
<p className="clinical-chat__load-title">{loadTitle}</p>
|
||||
<p className="clinical-chat__load-subtitle">{loadSubtitle}</p>
|
||||
<div className="clinical-chat__load-progress-row">
|
||||
<div className="clinical-chat__load-progress-track">
|
||||
<div
|
||||
className={`clinical-chat__load-progress-fill clinical-chat__load-progress-fill--${modelLoadPhase}`}
|
||||
className={`clinical-chat__load-progress-fill clinical-chat__load-progress-fill--${modelLoadPhase} ${isInstallStalled ? 'clinical-chat__load-progress-fill--stalled' : ''}`}
|
||||
style={{ width: `${progressPercent}%` }}
|
||||
/>
|
||||
</div>
|
||||
<span className="clinical-chat__load-progress-label tnum">{installProgressLabel}</span>
|
||||
<span className={`clinical-chat__load-progress-label tnum ${isInstallStalled ? 'clinical-chat__load-progress-label--stalled' : ''}`}>{installProgressLabel}</span>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
@@ -781,12 +815,43 @@ const styles = `
|
||||
color: var(--color-secondary);
|
||||
}
|
||||
.clinical-chat__load-icon--installing-gemma,
|
||||
.clinical-chat__load-icon--resuming-gemma,
|
||||
.clinical-chat__load-icon--installing-qwen {
|
||||
animation: clinical-chat-spin 2.4s linear infinite;
|
||||
}
|
||||
.clinical-chat__load-icon--installing {
|
||||
animation: clinical-chat-spin 2.4s linear infinite;
|
||||
}
|
||||
/* Stalled: stop the spin so a frozen download never looks like active progress. */
|
||||
.clinical-chat__load-bubble--stalled {
|
||||
border-color: rgba(180, 120, 40, 0.4);
|
||||
}
|
||||
.clinical-chat__load-icon--stalled {
|
||||
animation: clinical-chat-stall-pulse 1.6s ease-in-out infinite;
|
||||
background: rgba(180, 120, 40, 0.14);
|
||||
color: #9a6b1f;
|
||||
}
|
||||
@keyframes clinical-chat-stall-pulse {
|
||||
0%, 100% { opacity: 0.55; }
|
||||
50% { opacity: 1; }
|
||||
}
|
||||
.clinical-chat__load-progress-fill--stalled {
|
||||
background: repeating-linear-gradient(
|
||||
45deg,
|
||||
rgba(180, 120, 40, 0.55),
|
||||
rgba(180, 120, 40, 0.55) 6px,
|
||||
rgba(180, 120, 40, 0.3) 6px,
|
||||
rgba(180, 120, 40, 0.3) 12px
|
||||
);
|
||||
animation: clinical-chat-stall-stripes 0.8s linear infinite;
|
||||
}
|
||||
@keyframes clinical-chat-stall-stripes {
|
||||
from { background-position: 0 0; }
|
||||
to { background-position: 17px 0; }
|
||||
}
|
||||
.clinical-chat__load-progress-label--stalled {
|
||||
color: #9a6b1f;
|
||||
}
|
||||
@keyframes clinical-chat-pulse-icon {
|
||||
0%, 100% { opacity: 0.55; transform: scale(0.94); }
|
||||
50% { opacity: 1; transform: scale(1); }
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { memo, useEffect, useId, useLayoutEffect, useRef, useState } from 'react';
|
||||
import { memo, useEffect, useId, useRef, useState } from 'react';
|
||||
import StreamingPlainText from '../atoms/StreamingPlainText';
|
||||
import { streamTargetKey } from '../../lib/llm/clinicalChatStreamRegistry';
|
||||
|
||||
@@ -18,26 +18,14 @@ function ClinicalChatThought({
|
||||
thoughtStreaming = false,
|
||||
}: ClinicalChatThoughtProps) {
|
||||
const panelId = useId();
|
||||
const wasThoughtStreamingRef = useRef(thoughtStreaming);
|
||||
const userToggledRef = useRef(false);
|
||||
const [expanded, setExpanded] = useState(true);
|
||||
// Collapsed by default — keeps the clinical answer prominent; click to expand.
|
||||
const [expanded, setExpanded] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
setExpanded(true);
|
||||
userToggledRef.current = false;
|
||||
wasThoughtStreamingRef.current = false;
|
||||
}, [messageId]);
|
||||
|
||||
useLayoutEffect(() => {
|
||||
if (thoughtStreaming) {
|
||||
if (!userToggledRef.current) {
|
||||
setExpanded(true);
|
||||
}
|
||||
} else if (wasThoughtStreamingRef.current && !thoughtStreaming && !userToggledRef.current) {
|
||||
setExpanded(false);
|
||||
}
|
||||
wasThoughtStreamingRef.current = thoughtStreaming;
|
||||
}, [thoughtStreaming]);
|
||||
userToggledRef.current = false;
|
||||
}, [messageId]);
|
||||
|
||||
if (!content.trim() && !thoughtStreaming) {
|
||||
return null;
|
||||
@@ -49,7 +37,8 @@ function ClinicalChatThought({
|
||||
};
|
||||
|
||||
const label = thoughtStreaming ? 'Đang suy luận' : 'Suy luận';
|
||||
const preview = !thoughtStreaming && !expanded ? content.replace(/\s+/g, ' ').trim().slice(0, 100) : '';
|
||||
const preview =
|
||||
!expanded && !thoughtStreaming ? content.replace(/\s+/g, ' ').trim().slice(0, 100) : '';
|
||||
|
||||
return (
|
||||
<div
|
||||
@@ -77,15 +66,15 @@ function ClinicalChatThought({
|
||||
{!expanded && preview ? (
|
||||
<p className="clinical-chat__thought-preview">{preview}…</p>
|
||||
) : null}
|
||||
{expanded ? (
|
||||
<div id={panelId} className="clinical-chat__thought-body">
|
||||
{/* Body stays mounted even when collapsed so the imperative stream target keeps
|
||||
receiving tokens; visibility is toggled via the hidden attribute. */}
|
||||
<div id={panelId} className="clinical-chat__thought-body" hidden={!expanded}>
|
||||
<StreamingPlainText
|
||||
text={content}
|
||||
streamTargetKey={streamTargetKey(messageId, 'thought')}
|
||||
className="clinical-chat__thought-md chat-md__plain"
|
||||
/>
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -48,32 +48,35 @@ export default function RecordingModeSelector({ value, onChange, disabled }: Rec
|
||||
padding: 0;
|
||||
}
|
||||
.recording-mode-selector legend {
|
||||
font-size: 11px;
|
||||
font-weight: 600;
|
||||
color: #94a3b8;
|
||||
margin-bottom: 6px;
|
||||
font-size: 13px;
|
||||
font-weight: 700;
|
||||
color: #e2e8f0;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
.recording-mode-selector__options {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
gap: 8px;
|
||||
}
|
||||
.recording-mode-selector__option {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
font-size: 12px;
|
||||
color: #e2e8f0;
|
||||
gap: 10px;
|
||||
font-size: 14px;
|
||||
font-weight: 500;
|
||||
color: #f1f5f9;
|
||||
cursor: pointer;
|
||||
}
|
||||
.recording-mode-selector__option input {
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
accent-color: #76c8b1;
|
||||
}
|
||||
.recording-mode-selector__hint {
|
||||
margin: 6px 0 0;
|
||||
font-size: 11px;
|
||||
line-height: 1.45;
|
||||
color: #64748b;
|
||||
margin: 8px 0 0;
|
||||
font-size: 12.5px;
|
||||
line-height: 1.5;
|
||||
color: #cbd5e1;
|
||||
}
|
||||
.recording-mode-selector:disabled .recording-mode-selector__option {
|
||||
opacity: 0.55;
|
||||
|
||||
@@ -71,7 +71,7 @@ export default function SeverityBadge({
|
||||
<div className="severity-panel">
|
||||
{severityLoading ? (
|
||||
<div className="severity-panel__block glass">
|
||||
<span className="severity-panel__eyebrow">Viêm màng hoạt dịch — độ nặng (từ phân đoạn)</span>
|
||||
<span className="severity-panel__eyebrow">Viêm màng hoạt dịch — độ nặng (từ ảnh phân đoạn)</span>
|
||||
<p className="severity-panel__pending">Đang chờ kết quả độ nặng từ phân đoạn…</p>
|
||||
</div>
|
||||
) : grade != null && gradeLabel ? (
|
||||
@@ -80,7 +80,7 @@ export default function SeverityBadge({
|
||||
{grade}
|
||||
</span>
|
||||
<div>
|
||||
<span className="severity-badge__label">Viêm màng hoạt dịch — độ nặng (từ phân đoạn)</span>
|
||||
<span className="severity-badge__label">Viêm màng hoạt dịch — độ nặng (từ ảnh phân đoạn)</span>
|
||||
<strong>{gradeLabel}</strong>
|
||||
{severity?.description && (
|
||||
<p className="severity-panel__desc">{severity.description}</p>
|
||||
|
||||
@@ -34,6 +34,7 @@ interface DiagnosticCanvasProps {
|
||||
patientMrn?: string;
|
||||
patientId?: string;
|
||||
scanFrames?: ScanFrame[];
|
||||
useCelery?: boolean;
|
||||
}
|
||||
|
||||
export default function DiagnosticCanvas({
|
||||
@@ -54,6 +55,7 @@ export default function DiagnosticCanvas({
|
||||
patientMrn,
|
||||
patientId,
|
||||
scanFrames: scanFramesProp,
|
||||
useCelery,
|
||||
}: DiagnosticCanvasProps) {
|
||||
const activeFrames =
|
||||
(scanFramesProp && scanFramesProp.length > 0)
|
||||
@@ -146,7 +148,7 @@ export default function DiagnosticCanvas({
|
||||
const lockFrameNav = isSingleFrameNavLocked;
|
||||
|
||||
const { overlaySrc, interpretation, angleClassification, inflammationClassification, synovitisSeverity, isLoading, isSegmentationLoading, error, source, retry } =
|
||||
useSegmentationOverlay(frame.id, frame.src, framesForCanvas, patientMrn);
|
||||
useSegmentationOverlay(frame.id, frame.src, framesForCanvas, patientMrn, useCelery);
|
||||
|
||||
useEffect(() => {
|
||||
applyFrameIndex(0);
|
||||
@@ -360,13 +362,6 @@ export default function DiagnosticCanvas({
|
||||
{showMask && (
|
||||
<SegmentationOverlay overlaySrc={overlaySrc} isLoading={isLoading} />
|
||||
)}
|
||||
{showMask && error && !isLoading && (
|
||||
<MlServiceErrorPanel
|
||||
error={error}
|
||||
frameLabel={frameLabel}
|
||||
onRetry={source === 'backend' ? retry : undefined}
|
||||
/>
|
||||
)}
|
||||
<canvas
|
||||
ref={canvasRef}
|
||||
className="diagnostic-canvas__annotation-canvas"
|
||||
@@ -377,6 +372,13 @@ export default function DiagnosticCanvas({
|
||||
{...drawHandlers}
|
||||
/>
|
||||
</div>
|
||||
{showMask && error && !isLoading && (
|
||||
<MlServiceErrorPanel
|
||||
error={error}
|
||||
frameLabel={frameLabel}
|
||||
onRetry={source === 'backend' ? retry : undefined}
|
||||
/>
|
||||
)}
|
||||
{closedLoopPrompt && closedLoopPromptViewportAnchor && (
|
||||
<ClosedLoopPrompt
|
||||
anchor={closedLoopPromptViewportAnchor}
|
||||
|
||||
@@ -249,7 +249,7 @@ export default function ReviewDiagnosticSessionPanel({
|
||||
return (
|
||||
<div className="review-session">
|
||||
<header className="review-session__header">
|
||||
<h2 className="review-session__title">Xem lại phiên chẩn đoán</h2>
|
||||
<h2 className="review-session__title">XEM LẠI PHIÊN CHUẨN ĐOÁN</h2>
|
||||
<RecordingModeSelector
|
||||
value={lifecycle.recordingMode}
|
||||
onChange={lifecycle.setRecordingMode}
|
||||
@@ -572,7 +572,7 @@ export default function ReviewDiagnosticSessionPanel({
|
||||
padding-bottom: 8px;
|
||||
}
|
||||
.review-session__title {
|
||||
margin: 0;
|
||||
margin: 0 0 16px;
|
||||
font-size: 15px;
|
||||
font-weight: 700;
|
||||
letter-spacing: 0.02em;
|
||||
|
||||
@@ -155,7 +155,7 @@ export default function SideNavBar({
|
||||
<>
|
||||
<aside className="side-nav glass-elevated">
|
||||
<section className="side-nav__section">
|
||||
<h3>Đề xuất AI</h3>
|
||||
<h3>Đề xuất của AI</h3>
|
||||
|
||||
{hasCalibratableOutput && (
|
||||
<CalibrationControls config={userConfig} onChange={setUserConfig} />
|
||||
|
||||
@@ -153,9 +153,11 @@ export default function WorkspaceShell({ zoneA, zoneB }: WorkspaceShellProps) {
|
||||
<style>{`
|
||||
.workspace-shell {
|
||||
display: flex;
|
||||
flex: 1;
|
||||
flex: 0 0 calc(100dvh - var(--topbar-h) - var(--bottombar-h));
|
||||
height: calc(100dvh - var(--topbar-h) - var(--bottombar-h));
|
||||
min-height: 0;
|
||||
padding: var(--space-md);
|
||||
overflow: hidden;
|
||||
user-select: ${isDragging ? 'none' : 'auto'};
|
||||
}
|
||||
.workspace-shell--dragging {
|
||||
@@ -164,8 +166,10 @@ export default function WorkspaceShell({ zoneA, zoneB }: WorkspaceShellProps) {
|
||||
.workspace-shell__zone-a {
|
||||
flex: 0 0 var(--workspace-zone-a-pct);
|
||||
min-width: 0;
|
||||
min-height: 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
overflow: hidden;
|
||||
transition: flex-basis 0.05s linear;
|
||||
}
|
||||
.workspace-shell--dragging .workspace-shell__zone-a {
|
||||
@@ -174,8 +178,10 @@ export default function WorkspaceShell({ zoneA, zoneB }: WorkspaceShellProps) {
|
||||
.workspace-shell__zone-b {
|
||||
flex: 1 1 0;
|
||||
min-width: 0;
|
||||
min-height: 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
overflow: hidden;
|
||||
font-size: calc(1rem * var(--workspace-panel-scale, 1));
|
||||
}
|
||||
.workspace-shell__divider {
|
||||
@@ -246,13 +252,17 @@ export default function WorkspaceShell({ zoneA, zoneB }: WorkspaceShellProps) {
|
||||
}
|
||||
@media (max-width: 1024px) {
|
||||
.workspace-shell {
|
||||
flex: 1 1 auto;
|
||||
height: auto;
|
||||
flex-direction: column;
|
||||
gap: var(--space-md);
|
||||
overflow: visible;
|
||||
}
|
||||
.workspace-shell__zone-a,
|
||||
.workspace-shell__zone-b {
|
||||
flex: 1 1 auto;
|
||||
font-size: 1rem;
|
||||
overflow: visible;
|
||||
}
|
||||
}
|
||||
`}</style>
|
||||
|
||||
@@ -27,7 +27,7 @@ export const CALIBRATION_METRIC_HELP_KEY_POINTS: readonly CalibrationKeyPoint[]
|
||||
{
|
||||
title: 'Cơ chế bóp méo của AI hiện đại',
|
||||
body:
|
||||
'Các mạng càng sâu và thông minh thì xu hướng phân tách các điểm thô càng xa (ví dụ 20 điểm và 1 điểm). Khi quy đổi, khoảng cách quá lớn này bị ép thành xác suất cực đoan như 99,9% — hiện tượng “quá tự tin” (over-confidence).',
|
||||
'Các mô hình AI càng phức tạp (kích thuớc lớn & sâu) và thông minh thì xu hướng phân tách các điểm thô càng xa (ví dụ 20 điểm và 1 điểm). Khi quy đổi, khoảng cách quá lớn này bị ép thành xác suất cực đoan như 99,9% — hiện tượng “quá tự tin” (over-confidence).',
|
||||
},
|
||||
{
|
||||
title: 'Bác sĩ trưởng khoa hạ hỏa (tham số T)',
|
||||
|
||||
@@ -22,19 +22,19 @@ export const CALIBRATION_TIERS: CalibrationTier[] = [
|
||||
tier: 1,
|
||||
labelVi: 'Nhạy cao / Sàng lọc',
|
||||
labelEn: 'Aggressive / Screening',
|
||||
triggerVi: 'Bác sĩ nghi ngờ viêm cao từ triệu chứng, chưa thấy trên ảnh.',
|
||||
ruleVi: 'T = 0.7 (Sharpening)',
|
||||
triggerVi: 'Bác sĩ nghi ngờ mức độ viêm cao từ triệu chứng, chưa thấy trên ảnh.',
|
||||
ruleVi: 'T = 0.7',
|
||||
recommendedT: 0.7,
|
||||
uiEffectVi:
|
||||
'Đẩy xác suất biên lên — hạ ngưỡng cảnh báo viêm để không bỏ sót ca ranh giới.',
|
||||
'Đẩy xác suất biên lên — hạ ngưỡng cảnh báo viêm để không bỏ sót ca hiếm gặp.',
|
||||
},
|
||||
{
|
||||
id: 'standard',
|
||||
tier: 2,
|
||||
labelVi: 'Chuẩn / Mặc định',
|
||||
labelEn: 'Standard Baseline',
|
||||
triggerVi: 'Vận hành mặc định — chưa có prior lâm sàng từ người dùng.',
|
||||
ruleVi: 'T = 1.4 (Smoothing)',
|
||||
triggerVi: 'Vận hành mặc định — chưa có dự đoán từ trước từ người dùng.',
|
||||
ruleVi: 'T = 1.4',
|
||||
recommendedT: 1.4,
|
||||
uiEffectVi:
|
||||
'Giảm overconfidence của mạng — phân bố thực tế, cân bằng toán học.',
|
||||
@@ -45,7 +45,7 @@ export const CALIBRATION_TIERS: CalibrationTier[] = [
|
||||
labelVi: 'Bảo thủ / Hoài nghi',
|
||||
labelEn: 'Conservative / Skeptical',
|
||||
triggerVi: 'Bác sĩ tin bệnh nhân khỏe; dùng AI chỉ để kiểm tra lại.',
|
||||
ruleVi: 'T = 2.2 (Heavy flattening)',
|
||||
ruleVi: 'T = 2.2',
|
||||
recommendedT: 2.2,
|
||||
uiEffectVi:
|
||||
'Làm phẳng phân bố — chỉ báo dương khi tín hiệu mô hình cực kỳ mạnh.',
|
||||
|
||||
@@ -84,12 +84,17 @@ export type ClinicalChatRuntime = 'loading' | 'llm' | 'mock';
|
||||
|
||||
export type ModelLoadPhase =
|
||||
| 'installing-gemma'
|
||||
| 'resuming-gemma'
|
||||
// | 'installing-qwen'
|
||||
| 'loading-gemma';
|
||||
// | 'loading-qwen';
|
||||
|
||||
/** No download progress for this long ⇒ treat the install as stalled (not "loading"). */
|
||||
export const MODEL_INSTALL_STALL_MS = 15_000;
|
||||
const MODEL_INSTALL_STALL_POLL_MS = 3_000;
|
||||
|
||||
export function isModelInstallPhase(phase: ModelLoadPhase): boolean {
|
||||
return phase === 'installing-gemma';
|
||||
return phase === 'installing-gemma' || phase === 'resuming-gemma';
|
||||
// || phase === 'installing-qwen';
|
||||
}
|
||||
|
||||
@@ -137,6 +142,8 @@ export interface UseClinicalChatResult {
|
||||
modelLoadProgress: number;
|
||||
/** Byte transfer label during OPFS install (e.g. "842.3 MB / 1.87 GB"). */
|
||||
modelInstallTransferLabel: string | null;
|
||||
/** True when an install/resume has received no bytes for a while — not actually progressing. */
|
||||
modelLoadStalled: boolean;
|
||||
modelLoadFading: boolean;
|
||||
sendMessage: () => void;
|
||||
stopGeneration: () => void;
|
||||
@@ -212,6 +219,7 @@ export function useClinicalChat({
|
||||
const [modelLoadPhase, setModelLoadPhase] = useState<ModelLoadPhase | null>(null);
|
||||
const [modelLoadProgress, setModelLoadProgress] = useState(0);
|
||||
const [modelInstallTransferLabel, setModelInstallTransferLabel] = useState<string | null>(null);
|
||||
const [modelLoadStalled, setModelLoadStalled] = useState(false);
|
||||
const [modelLoadFading, setModelLoadFading] = useState(false);
|
||||
const [modeSuggestion, setModeSuggestion] = useState<ModeSuggestion | null>(null);
|
||||
|
||||
@@ -365,11 +373,49 @@ export function useClinicalChat({
|
||||
let cancelled = false;
|
||||
let fadeTimer: number | undefined;
|
||||
let stopLoadTicker: (() => void) | undefined;
|
||||
let stallWatchdogId: number | undefined;
|
||||
let lastInstallProgressAt = Date.now();
|
||||
// Highest byte offset seen so far. Retry "heartbeats" re-emit the same offset;
|
||||
// only a real increase counts as progress.
|
||||
let lastInstallBytes = -1;
|
||||
|
||||
const clearStallWatchdog = () => {
|
||||
if (stallWatchdogId !== undefined) {
|
||||
window.clearInterval(stallWatchdogId);
|
||||
stallWatchdogId = undefined;
|
||||
}
|
||||
};
|
||||
|
||||
// Flip the overlay from "downloading" to "stalled" when no *new bytes* arrive for a
|
||||
// while, so a frozen bar (or a resume stuck retrying the same offset) never
|
||||
// masquerades as active progress.
|
||||
const startStallWatchdog = () => {
|
||||
clearStallWatchdog();
|
||||
lastInstallProgressAt = Date.now();
|
||||
lastInstallBytes = -1;
|
||||
setModelLoadStalled(false);
|
||||
stallWatchdogId = window.setInterval(() => {
|
||||
if (cancelled) {
|
||||
return;
|
||||
}
|
||||
if (Date.now() - lastInstallProgressAt > MODEL_INSTALL_STALL_MS) {
|
||||
setModelLoadStalled(true);
|
||||
setStatusLabel('Tải Gemma bị gián đoạn — đang thử kết nối lại…');
|
||||
}
|
||||
}, MODEL_INSTALL_STALL_POLL_MS);
|
||||
};
|
||||
|
||||
const noteInstallProgress = () => {
|
||||
lastInstallProgressAt = Date.now();
|
||||
setModelLoadStalled(false);
|
||||
};
|
||||
|
||||
async function finishModelLoad(): Promise<void> {
|
||||
if (cancelled) {
|
||||
return;
|
||||
}
|
||||
clearStallWatchdog();
|
||||
setModelLoadStalled(false);
|
||||
setModelLoadProgress(100);
|
||||
setModelLoadFading(true);
|
||||
await new Promise<void>((resolve) => {
|
||||
@@ -469,9 +515,25 @@ export function useClinicalChat({
|
||||
if (!cancelled) {
|
||||
setModelLoadProgress(8);
|
||||
if (!initialGemma.loadable) {
|
||||
setModelLoadPhase('installing-gemma');
|
||||
// A partial checkpoint already on disk means we resume, not start fresh.
|
||||
const isPartialResume = initialGemma.bytes > 0;
|
||||
setModelLoadPhase(isPartialResume ? 'resuming-gemma' : 'installing-gemma');
|
||||
setIsModelLoading(true);
|
||||
setStatusLabel('Đang cài đặt Gemma 4 E2B về máy…');
|
||||
setStatusLabel(
|
||||
isPartialResume
|
||||
? 'Đang tiếp tục tải Gemma 4 E2B…'
|
||||
: 'Đang cài đặt Gemma 4 E2B về máy…',
|
||||
);
|
||||
if (isPartialResume) {
|
||||
setModelInstallTransferLabel(
|
||||
formatInstallTransferLabel({
|
||||
phase: 'resuming',
|
||||
bytesLoaded: initialGemma.bytes,
|
||||
bytesTotal: initialGemma.manifest?.bytes ?? null,
|
||||
}),
|
||||
);
|
||||
}
|
||||
startStallWatchdog();
|
||||
}
|
||||
// else if (!initialQwen.loadable) {
|
||||
// setModelLoadPhase('installing-qwen');
|
||||
@@ -486,11 +548,26 @@ export function useClinicalChat({
|
||||
if (cancelled) {
|
||||
return;
|
||||
}
|
||||
setModelLoadPhase('installing-gemma');
|
||||
setIsModelLoading(true);
|
||||
setStatusLabel('Đang cài đặt Gemma 4 E2B về máy…');
|
||||
setModelInstallTransferLabel(formatInstallTransferLabel(progress));
|
||||
setModelLoadProgress(mapDownloadProgress(progress));
|
||||
|
||||
// Retry heartbeats re-emit the same offset — those must NOT reset the
|
||||
// watchdog or clear the stalled state, otherwise a stuck resume keeps
|
||||
// looking like active loading.
|
||||
const advanced = progress.bytesLoaded > lastInstallBytes;
|
||||
if (!advanced) {
|
||||
return;
|
||||
}
|
||||
lastInstallBytes = progress.bytesLoaded;
|
||||
noteInstallProgress();
|
||||
const resuming = progress.phase === 'resuming';
|
||||
setModelLoadPhase(resuming ? 'resuming-gemma' : 'installing-gemma');
|
||||
setStatusLabel(
|
||||
resuming
|
||||
? 'Đang tiếp tục tải Gemma 4 E2B…'
|
||||
: 'Đang cài đặt Gemma 4 E2B về máy…',
|
||||
);
|
||||
},
|
||||
// onQwenDownloadProgress: (progress: QwenDownloadProgress) => {
|
||||
// if (cancelled) {
|
||||
@@ -537,17 +614,20 @@ export function useClinicalChat({
|
||||
);
|
||||
void preloadGemmaIntoMemory();
|
||||
} catch (error) {
|
||||
clearStallWatchdog();
|
||||
if (!cancelled) {
|
||||
setIsModelLoading(false);
|
||||
setModelLoadFading(false);
|
||||
setModelLoadPhase(null);
|
||||
setModelInstallTransferLabel(null);
|
||||
setModelLoadStalled(false);
|
||||
setRuntime('mock');
|
||||
const message = error instanceof Error ? error.message : 'không tải được mô hình';
|
||||
const isNetwork =
|
||||
error instanceof TypeError ||
|
||||
(error instanceof DOMException && error.name === 'NetworkError') ||
|
||||
/network|failed to fetch|interrupted|gián đoạn|network_changed|err_network_changed/i.test(
|
||||
(error instanceof DOMException &&
|
||||
(error.name === 'NetworkError' || error.name === 'TimeoutError')) ||
|
||||
/network|failed to fetch|interrupted|timed out|timeout|gián đoạn|network_changed|err_network_changed/i.test(
|
||||
message,
|
||||
);
|
||||
setStatusLabel(
|
||||
@@ -563,6 +643,7 @@ export function useClinicalChat({
|
||||
return () => {
|
||||
cancelled = true;
|
||||
stopLoadTicker?.();
|
||||
clearStallWatchdog();
|
||||
if (fadeTimer !== undefined) {
|
||||
window.clearTimeout(fadeTimer);
|
||||
}
|
||||
@@ -765,22 +846,26 @@ export function useClinicalChat({
|
||||
let ollamaThoughtAcc = '';
|
||||
let ollamaAnswerAcc = '';
|
||||
|
||||
const assistantId = createChatMessageId();
|
||||
const assistantMessage: ClinicalChatMessage = {
|
||||
id: assistantId,
|
||||
// A single generation may spawn continuation segments; each segment renders as
|
||||
// its own bubble with its own reasoning, so continuation thinking never leaks
|
||||
// into a prior answer. currentAssistantId tracks the bubble being streamed.
|
||||
let currentAssistantId = createChatMessageId();
|
||||
let segmentCount = 1;
|
||||
const spawnAssistantBubble = (id: string, pondering: boolean): ClinicalChatMessage => ({
|
||||
id,
|
||||
role: 'assistant',
|
||||
content: '',
|
||||
timestamp: new Date(),
|
||||
streaming: true,
|
||||
tracksThought: thoughtActive,
|
||||
pondering: useRemote,
|
||||
pondering,
|
||||
ponderingVariant: mode === 'agent' ? 'agent' : 'chat',
|
||||
};
|
||||
setMessages((prev) => [...prev, assistantMessage]);
|
||||
});
|
||||
setMessages((prev) => [...prev, spawnAssistantBubble(currentAssistantId, useRemote)]);
|
||||
setStatusLabel(generationStatusLabel(mode, activeLevel));
|
||||
|
||||
let plainContentAccumulator = '';
|
||||
const thoughtParser =
|
||||
let thoughtParser =
|
||||
thoughtActive && !useOllamaThoughtStream ? createCotStreamParser() : null;
|
||||
let thoughtCompleteEmitted = false;
|
||||
let ponderingCleared = false;
|
||||
@@ -790,7 +875,7 @@ export function useClinicalChat({
|
||||
return;
|
||||
}
|
||||
ponderingCleared = true;
|
||||
updateMessage(assistantId, { pondering: false });
|
||||
updateMessage(currentAssistantId, { pondering: false });
|
||||
};
|
||||
|
||||
const useImperativeStreamPaint = useRemote || thoughtActive;
|
||||
@@ -801,8 +886,9 @@ export function useClinicalChat({
|
||||
thoughtComplete?: boolean;
|
||||
tracksThought?: boolean;
|
||||
}>((patch) => {
|
||||
const id = currentAssistantId;
|
||||
setMessages((prev) =>
|
||||
prev.map((msg) => (msg.id === assistantId ? { ...msg, ...patch } : msg)),
|
||||
prev.map((msg) => (msg.id === id ? { ...msg, ...patch } : msg)),
|
||||
);
|
||||
});
|
||||
|
||||
@@ -814,6 +900,7 @@ export function useClinicalChat({
|
||||
tracksThought?: boolean;
|
||||
},
|
||||
) => {
|
||||
const id = currentAssistantId;
|
||||
const hasStreamText =
|
||||
patch.thoughtContent !== undefined || patch.content !== undefined;
|
||||
|
||||
@@ -822,13 +909,13 @@ export function useClinicalChat({
|
||||
if (patch.thoughtContent !== undefined) {
|
||||
domHandled =
|
||||
setClinicalStreamText(
|
||||
streamTargetKey(assistantId, 'thought'),
|
||||
streamTargetKey(id, 'thought'),
|
||||
patch.thoughtContent,
|
||||
) && domHandled;
|
||||
}
|
||||
if (patch.content !== undefined) {
|
||||
domHandled =
|
||||
setClinicalStreamText(streamTargetKey(assistantId, 'answer'), patch.content) &&
|
||||
setClinicalStreamText(streamTargetKey(id, 'answer'), patch.content) &&
|
||||
domHandled;
|
||||
}
|
||||
|
||||
@@ -840,7 +927,7 @@ export function useClinicalChat({
|
||||
if (needsReact) {
|
||||
flushSync(() => {
|
||||
setMessages((prev) =>
|
||||
prev.map((msg) => (msg.id === assistantId ? { ...msg, ...patch } : msg)),
|
||||
prev.map((msg) => (msg.id === id ? { ...msg, ...patch } : msg)),
|
||||
);
|
||||
});
|
||||
}
|
||||
@@ -861,6 +948,45 @@ export function useClinicalChat({
|
||||
}
|
||||
};
|
||||
|
||||
// Freeze the bubble being streamed using its OWN parser/accumulator, so its
|
||||
// reasoning + answer slice stay self-contained.
|
||||
const finalizeCurrentBubble = () => {
|
||||
if (thoughtParser) {
|
||||
const snapshot = thoughtParser.finalize();
|
||||
updateMessage(currentAssistantId, {
|
||||
content: snapshot.content,
|
||||
thoughtContent: snapshot.thoughtContent || undefined,
|
||||
tracksThought: true,
|
||||
thoughtComplete: true,
|
||||
streaming: false,
|
||||
pondering: false,
|
||||
});
|
||||
} else {
|
||||
updateMessage(currentAssistantId, {
|
||||
content: plainContentAccumulator,
|
||||
streaming: false,
|
||||
pondering: false,
|
||||
});
|
||||
}
|
||||
clearClinicalStreamTargetsForMessage(currentAssistantId);
|
||||
};
|
||||
|
||||
// Continuation → close the current bubble and open a fresh one for the next segment.
|
||||
const startNextSegmentBubble = () => {
|
||||
finalizeCurrentBubble();
|
||||
if (!useImperativeStreamPaint) {
|
||||
edgeStreamRaf.cancel();
|
||||
}
|
||||
const nextId = createChatMessageId();
|
||||
currentAssistantId = nextId;
|
||||
segmentCount += 1;
|
||||
thoughtParser =
|
||||
thoughtActive && !useOllamaThoughtStream ? createCotStreamParser() : null;
|
||||
plainContentAccumulator = '';
|
||||
thoughtCompleteEmitted = false;
|
||||
setMessages((prev) => [...prev, spawnAssistantBubble(nextId, false)]);
|
||||
};
|
||||
|
||||
try {
|
||||
const runTurn = () =>
|
||||
runClinicalChatTurn(
|
||||
@@ -884,6 +1010,13 @@ export function useClinicalChat({
|
||||
},
|
||||
(event: ClinicalChatStreamEvent) => {
|
||||
try {
|
||||
if (event.type === 'segment_boundary') {
|
||||
// segment 1 is the bubble we already opened; only continuations spawn a new one.
|
||||
if (event.segment >= 2 && !useOllamaThoughtStream) {
|
||||
startNextSegmentBubble();
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (event.type === 'thought_token') {
|
||||
if (!event.partial || !useOllamaThoughtStream) {
|
||||
return;
|
||||
@@ -955,15 +1088,15 @@ export function useClinicalChat({
|
||||
|
||||
if (abortController.signal.aborted) {
|
||||
disposeStreamThrottle();
|
||||
updateMessage(assistantId, { streaming: false });
|
||||
updateMessage(currentAssistantId, { streaming: false });
|
||||
return;
|
||||
}
|
||||
|
||||
disposeStreamThrottle();
|
||||
clearClinicalStreamTargetsForMessage(assistantId);
|
||||
clearClinicalStreamTargetsForMessage(currentAssistantId);
|
||||
|
||||
if (useOllamaThoughtStream) {
|
||||
updateMessage(assistantId, {
|
||||
updateMessage(currentAssistantId, {
|
||||
content: ollamaAnswerAcc || result.finalAnswer,
|
||||
thoughtContent: ollamaThoughtAcc || undefined,
|
||||
tracksThought: true,
|
||||
@@ -973,8 +1106,11 @@ export function useClinicalChat({
|
||||
});
|
||||
} else if (thoughtActive && thoughtParser) {
|
||||
const snapshot = thoughtParser.finalize();
|
||||
const finalContent = snapshot.content || result.finalAnswer;
|
||||
updateMessage(assistantId, {
|
||||
// Only the sole segment may borrow the merged finalAnswer; continuation
|
||||
// bubbles must keep just their own parsed slice to avoid duplication.
|
||||
const finalContent =
|
||||
snapshot.content || (segmentCount === 1 ? result.finalAnswer : '');
|
||||
updateMessage(currentAssistantId, {
|
||||
content: finalContent,
|
||||
thoughtContent: snapshot.thoughtContent || undefined,
|
||||
tracksThought: true,
|
||||
@@ -983,8 +1119,9 @@ export function useClinicalChat({
|
||||
pondering: false,
|
||||
});
|
||||
} else {
|
||||
updateMessage(assistantId, {
|
||||
content: result.finalAnswer,
|
||||
updateMessage(currentAssistantId, {
|
||||
content:
|
||||
plainContentAccumulator || (segmentCount === 1 ? result.finalAnswer : ''),
|
||||
streaming: false,
|
||||
pondering: false,
|
||||
});
|
||||
@@ -999,12 +1136,12 @@ export function useClinicalChat({
|
||||
);
|
||||
} catch (error) {
|
||||
disposeStreamThrottle();
|
||||
clearClinicalStreamTargetsForMessage(assistantId);
|
||||
clearClinicalStreamTargetsForMessage(currentAssistantId);
|
||||
if (error instanceof DOMException && error.name === 'AbortError') {
|
||||
updateMessage(assistantId, { streaming: false, pondering: false });
|
||||
updateMessage(currentAssistantId, { streaming: false, pondering: false });
|
||||
return;
|
||||
}
|
||||
updateMessage(assistantId, {
|
||||
updateMessage(currentAssistantId, {
|
||||
content:
|
||||
error instanceof Error
|
||||
? `Không thể trả lời: ${error.message}`
|
||||
@@ -1132,6 +1269,7 @@ export function useClinicalChat({
|
||||
modelLoadPhase,
|
||||
modelLoadProgress,
|
||||
modelInstallTransferLabel,
|
||||
modelLoadStalled,
|
||||
modelLoadFading,
|
||||
sendMessage,
|
||||
stopGeneration,
|
||||
|
||||
@@ -12,7 +12,10 @@ import {
|
||||
normalizeBackendSeverity,
|
||||
type SynovitisSeverityResult,
|
||||
} from '../data/synovitisSeverity';
|
||||
import { getCachedCvAnalyzeResult, getCvAnalyzeResultsForProfile } from '../lib/cvAnalyzeApi';
|
||||
import { getCachedCvAnalyzeResult, getCvAnalyzeResultsForProfile, getCvAnalyzeResultsForProfileCelery, clearCvAnalyzeResultCache, type CvFrameAnalyzeResult } from '../lib/cvAnalyzeApi';
|
||||
import { clearSegmentationResultCache } from '../lib/segmentationApi';
|
||||
import { clearAngleClassificationResultCache } from '../lib/angleClassificationApi';
|
||||
import { clearMlInferenceCacheForPatient } from '../lib/mlInferenceCacheStore';
|
||||
import type { ProfileMlContext } from '../lib/mlInferenceCacheKeys';
|
||||
import type { ScanFrame } from '../data/scanFrames';
|
||||
import { interpretSegmentationForDisplay } from '../lib/interpretSegmentationResult';
|
||||
@@ -104,6 +107,7 @@ export function useSegmentationOverlay(
|
||||
imageSrc: string,
|
||||
profileFrames?: ScanFrame[],
|
||||
patientMrn?: string,
|
||||
useCelery?: boolean,
|
||||
): UseSegmentationOverlayResult {
|
||||
const [overlaySrc, setOverlaySrc] = useState<string | null>(null);
|
||||
const [interpretation, setInterpretation] = useState<SegmentationDisplayInterpretation | null>(null);
|
||||
@@ -117,7 +121,15 @@ export function useSegmentationOverlay(
|
||||
const [source, setSource] = useState<'backend' | null>(null);
|
||||
const [retryNonce, setRetryNonce] = useState(0);
|
||||
|
||||
const retry = () => setRetryNonce((n) => n + 1);
|
||||
const retry = () => {
|
||||
clearCvAnalyzeResultCache();
|
||||
clearSegmentationResultCache();
|
||||
clearAngleClassificationResultCache();
|
||||
if (patientMrn) {
|
||||
clearMlInferenceCacheForPatient(patientMrn);
|
||||
}
|
||||
setRetryNonce((n) => n + 1);
|
||||
};
|
||||
|
||||
const latestSegmentationRef = useRef<SegmentationApiResult | undefined>(undefined);
|
||||
|
||||
@@ -161,7 +173,14 @@ export function useSegmentationOverlay(
|
||||
const mlContext: ProfileMlContext | undefined = patientMrn
|
||||
? { patientMrn }
|
||||
: undefined;
|
||||
const results = await getCvAnalyzeResultsForProfile(frameRefs, mlContext);
|
||||
|
||||
let results: Map<string, CvFrameAnalyzeResult>;
|
||||
if (useCelery) {
|
||||
results = await getCvAnalyzeResultsForProfileCelery(frameRefs, mlContext);
|
||||
} else {
|
||||
results = await getCvAnalyzeResultsForProfile(frameRefs, mlContext);
|
||||
}
|
||||
|
||||
if (cancelled) return;
|
||||
|
||||
const cvResult = results.get(frameId);
|
||||
@@ -235,7 +254,7 @@ export function useSegmentationOverlay(
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [frameId, imageSrc, profileFrames, patientMrn, retryNonce]);
|
||||
}, [frameId, imageSrc, profileFrames, patientMrn, retryNonce, useCelery]);
|
||||
|
||||
return {
|
||||
overlaySrc,
|
||||
|
||||
@@ -31,6 +31,26 @@ export interface BackendCvAnalyzeBatchResponse {
|
||||
detail?: string;
|
||||
}
|
||||
|
||||
export interface BackendCvCelerySubmitResponse {
|
||||
success: boolean;
|
||||
job_id: string;
|
||||
image_count: number;
|
||||
mode: string;
|
||||
chunk_size: number;
|
||||
detail?: string;
|
||||
}
|
||||
|
||||
export interface BackendCvCeleryStatusResponse {
|
||||
status: 'pending' | 'completed' | 'failed' | 'unknown';
|
||||
completed?: number;
|
||||
total?: number;
|
||||
progress?: number;
|
||||
image_count?: number;
|
||||
results?: BackendSegmentationResponse[];
|
||||
errors?: string[];
|
||||
detail?: string;
|
||||
}
|
||||
|
||||
export interface CvFrameAnalyzeResult {
|
||||
raw: BackendSegmentationResponse;
|
||||
segmentation: SegmentationApiResult;
|
||||
@@ -222,3 +242,166 @@ export async function getCvAnalyzeResultsForProfile(
|
||||
}
|
||||
return resolved;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Submit CV batch for async Celery chunk fan-out.
|
||||
* Returns job_id immediately — poll with pollCvAnalyzeBatchCelery().
|
||||
*/
|
||||
export async function submitCvAnalyzeBatchCelery(
|
||||
frames: ProfileFrameRef[],
|
||||
apiBase = getSegmentApiBase(),
|
||||
): Promise<string> {
|
||||
if (frames.length === 0) {
|
||||
throw new Error('frames must not be empty');
|
||||
}
|
||||
|
||||
const formData = new FormData();
|
||||
const files = await Promise.all(
|
||||
frames.map((frame, index) =>
|
||||
imageUrlToFile(frame.src, `${frame.id || `frame-${index}`}.png`),
|
||||
),
|
||||
);
|
||||
files.forEach((file) => formData.append('images', file));
|
||||
formData.append('frame_ids', JSON.stringify(frames.map((frame) => frame.id)));
|
||||
|
||||
const response = await fetch(`${apiBase}/api/test/analyze/batch/celery`, {
|
||||
method: 'POST',
|
||||
body: formData,
|
||||
});
|
||||
|
||||
const payload = await readMlApiJson<BackendCvCelerySubmitResponse>(response);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Celery batch submit error (${response.status})`);
|
||||
}
|
||||
|
||||
if (!payload.success || !payload.job_id) {
|
||||
throw new Error('Celery batch submit returned success=false or missing job_id');
|
||||
}
|
||||
|
||||
return payload.job_id;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Poll Celery batch job status.
|
||||
* Returns status object — call again if status === 'pending'.
|
||||
*/
|
||||
export async function pollCvAnalyzeBatchCelery(
|
||||
jobId: string,
|
||||
apiBase = getSegmentApiBase(),
|
||||
): Promise<BackendCvCeleryStatusResponse> {
|
||||
const response = await fetch(`${apiBase}/api/test/analyze/batch/celery/${encodeURIComponent(jobId)}`);
|
||||
|
||||
const payload = await readMlApiJson<BackendCvCeleryStatusResponse>(response);
|
||||
if (!response.ok) {
|
||||
throw new Error(payload.detail ?? `Celery batch poll error (${response.status})`);
|
||||
}
|
||||
|
||||
return payload;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Async CV pipeline via Celery chunk fan-out.
|
||||
* Submits job, polls until completed/failed, maps results into cache.
|
||||
* Returns cached results immediately if all frames are already available.
|
||||
*/
|
||||
export async function getCvAnalyzeResultsForProfileCelery(
|
||||
frames: ProfileFrameRef[],
|
||||
mlContext?: ProfileMlContext,
|
||||
apiBase = getSegmentApiBase(),
|
||||
signal?: AbortSignal,
|
||||
): Promise<Map<string, CvFrameAnalyzeResult>> {
|
||||
if (frames.length === 0) {
|
||||
return new Map();
|
||||
}
|
||||
|
||||
const cacheKey = buildBatchCacheKey(frames.map((f) => f.id));
|
||||
|
||||
// Return fully cached results without hitting the backend.
|
||||
const cached = new Map<string, CvFrameAnalyzeResult>();
|
||||
for (const frame of frames) {
|
||||
const hit = cvAnalyzeResultCache.get(frame.id);
|
||||
if (hit) {
|
||||
cached.set(frame.id, hit);
|
||||
}
|
||||
}
|
||||
if (cached.size === frames.length) {
|
||||
return cached;
|
||||
}
|
||||
|
||||
// Coalesce in-flight requests for the same batch of frames.
|
||||
const existing = inflightBatchByKey.get(cacheKey);
|
||||
if (existing) {
|
||||
return existing;
|
||||
}
|
||||
|
||||
const batchPromise = (async (): Promise<Map<string, CvFrameAnalyzeResult>> => {
|
||||
const jobId = await submitCvAnalyzeBatchCelery(frames, apiBase);
|
||||
const pollIntervalMs = 2000;
|
||||
const submitTime = Date.now();
|
||||
|
||||
while (true) {
|
||||
if (signal?.aborted) {
|
||||
throw new Error('Celery batch poll aborted');
|
||||
}
|
||||
|
||||
const status = await pollCvAnalyzeBatchCelery(jobId, apiBase);
|
||||
|
||||
if (status.status === 'completed') {
|
||||
const byFrameId = new Map<string, CvFrameAnalyzeResult>();
|
||||
if (status.results) {
|
||||
for (const item of status.results) {
|
||||
const frameId = item.frame_id;
|
||||
if (!frameId) continue;
|
||||
const cvResult = mapPayloadToCvResult(item);
|
||||
byFrameId.set(frameId, cvResult);
|
||||
cvAnalyzeResultCache.set(frameId, cvResult);
|
||||
segmentationResultCache.set(frameId, cvResult.segmentation);
|
||||
if (cvResult.angle) {
|
||||
angleResultCache.set(frameId, cvResult.angle);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (mlContext?.patientMrn && status.results) {
|
||||
await persistCvBatch(
|
||||
frames,
|
||||
new Map([...byFrameId.entries()]),
|
||||
mlContext,
|
||||
);
|
||||
}
|
||||
const totalMs = Date.now() - submitTime;
|
||||
console.log(
|
||||
`[cvAnalyzeApi] Celery batch completed jobId=${jobId} frames=${frames.length} totalMs=${totalMs}`,
|
||||
);
|
||||
return byFrameId;
|
||||
}
|
||||
|
||||
if (status.status === 'failed') {
|
||||
const totalMs = Date.now() - submitTime;
|
||||
console.error(
|
||||
`[cvAnalyzeApi] Celery batch failed jobId=${jobId} frames=${frames.length} totalMs=${totalMs}`,
|
||||
);
|
||||
throw new Error(status.errors?.join('; ') ?? 'Celery batch job failed');
|
||||
}
|
||||
|
||||
if (status.status === 'unknown') {
|
||||
const totalMs = Date.now() - submitTime;
|
||||
console.error(
|
||||
`[cvAnalyzeApi] Celery batch unknown jobId=${jobId} frames=${frames.length} totalMs=${totalMs}`,
|
||||
);
|
||||
throw new Error(status.detail ?? `Celery job ${jobId} not found`);
|
||||
}
|
||||
|
||||
await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
|
||||
}
|
||||
})();
|
||||
|
||||
inflightBatchByKey.set(cacheKey, batchPromise);
|
||||
try {
|
||||
return await batchPromise;
|
||||
} finally {
|
||||
inflightBatchByKey.delete(cacheKey);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -24,12 +24,12 @@ function envFlag(name: string, defaultValue: boolean): boolean {
|
||||
|
||||
/** Try local Gemma worker when OPFS model is available. Falls back to mock replies otherwise. */
|
||||
export function useLocalLlmWhenAvailable(): boolean {
|
||||
return envFlag('VITE_CLINICAL_CHAT_USE_LLM', true);
|
||||
return import.meta.env.VITE_CLINICAL_CHAT_USE_LLM !== 'false';
|
||||
}
|
||||
|
||||
/** Use fixture tool results instead of BFF (default on in dev). */
|
||||
export function useMockAgentTools(): boolean {
|
||||
return envFlag('VITE_CLINICAL_CHAT_MOCK_TOOLS', true);
|
||||
return import.meta.env.VITE_CLINICAL_CHAT_MOCK_TOOLS !== 'false';
|
||||
}
|
||||
|
||||
export function clinicalChatBffBaseUrl(): string {
|
||||
|
||||
@@ -122,6 +122,7 @@ export class LlmWorkerClient {
|
||||
promptOptions: PromptOptions,
|
||||
decode: DecodeParams,
|
||||
onToken?: (partial: string) => void,
|
||||
onSegmentStart?: (segment: number) => void,
|
||||
): Promise<{ rawOutput: string; stats: GenerationStats }> {
|
||||
const id = requestId();
|
||||
this.activeGenerateRequestId = id;
|
||||
@@ -136,6 +137,7 @@ export class LlmWorkerClient {
|
||||
},
|
||||
reject,
|
||||
onToken: (partial) => onToken?.(partial),
|
||||
onSegmentStart: (segment) => onSegmentStart?.(segment),
|
||||
});
|
||||
this.worker.postMessage({
|
||||
type: 'generate',
|
||||
|
||||
@@ -16,19 +16,23 @@ export function formatInstallTransferLabel(progress: DownloadProgress): string {
|
||||
export function mapDownloadProgress(progress: DownloadProgress): number {
|
||||
switch (progress.phase) {
|
||||
case 'downloading':
|
||||
case 'resuming':
|
||||
case 'resuming': {
|
||||
if (!progress.bytesTotal || progress.bytesTotal <= 0) {
|
||||
return progress.phase === 'resuming' ? 14 : 12;
|
||||
return progress.phase === 'resuming' ? 6 : 4;
|
||||
}
|
||||
// Track the real byte fraction so the bar matches the "X GB / Y GB" label,
|
||||
// reserving the last few percent for verify/write before completion.
|
||||
const fraction = Math.min(1, progress.bytesLoaded / progress.bytesTotal);
|
||||
return 4 + Math.round(fraction * 92);
|
||||
}
|
||||
return 10 + Math.round((progress.bytesLoaded / progress.bytesTotal) * 58);
|
||||
case 'hashing':
|
||||
return 72;
|
||||
return 97;
|
||||
case 'writing':
|
||||
return 76;
|
||||
return 98;
|
||||
case 'done':
|
||||
return 78;
|
||||
return 99;
|
||||
default:
|
||||
return 10;
|
||||
return 4;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -66,12 +66,26 @@ interface DownloadProbe {
|
||||
supportsRange: boolean;
|
||||
}
|
||||
|
||||
/** HEAD/range probes should return fast; time out so a hung socket surfaces as a retriable error. */
|
||||
const PROBE_TIMEOUT_MS = 12_000;
|
||||
|
||||
function probeTimeoutSignal(): AbortSignal | undefined {
|
||||
return typeof AbortSignal !== 'undefined' && typeof AbortSignal.timeout === 'function'
|
||||
? AbortSignal.timeout(PROBE_TIMEOUT_MS)
|
||||
: undefined;
|
||||
}
|
||||
|
||||
async function probeModelDownloadUrl(url: string): Promise<DownloadProbe> {
|
||||
let response = await fetch(url, { method: 'HEAD', redirect: 'follow' });
|
||||
let response = await fetch(url, {
|
||||
method: 'HEAD',
|
||||
redirect: 'follow',
|
||||
signal: probeTimeoutSignal(),
|
||||
});
|
||||
if (!response.ok) {
|
||||
response = await fetch(url, {
|
||||
headers: { Range: 'bytes=0-0' },
|
||||
redirect: 'follow',
|
||||
signal: probeTimeoutSignal(),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -164,7 +178,7 @@ function isRetriableDownloadError(error: unknown): boolean {
|
||||
if (error instanceof TypeError) {
|
||||
return true;
|
||||
}
|
||||
if (error instanceof DOMException && error.name === 'NetworkError') {
|
||||
if (error instanceof DOMException && (error.name === 'NetworkError' || error.name === 'TimeoutError')) {
|
||||
return true;
|
||||
}
|
||||
const message = (error instanceof Error ? error.message : String(error)).toLowerCase();
|
||||
@@ -175,6 +189,8 @@ function isRetriableDownloadError(error: unknown): boolean {
|
||||
message.includes('failed to fetch') ||
|
||||
message.includes('load failed') ||
|
||||
message.includes('interrupted') ||
|
||||
message.includes('timed out') ||
|
||||
message.includes('timeout') ||
|
||||
message.includes('gián đoạn') ||
|
||||
message.includes('http 502') ||
|
||||
message.includes('http 503') ||
|
||||
@@ -328,6 +344,17 @@ export async function checkOpfsModelLoadable(): Promise<OpfsModelLoadableStatus>
|
||||
|
||||
const file = await readOpfsModelFile(MODEL_TASK_FILENAME);
|
||||
if (file && file.size > 0 && (await isReadable(file))) {
|
||||
// An interrupted download never wrote a manifest (it is written last). Such a
|
||||
// file can still clear the >500 MB header check while being a truncated,
|
||||
// unloadable checkpoint — treat it as resumable, not loadable, so we resume the
|
||||
// download instead of trying to init MediaPipe on a fragmented model.
|
||||
if (file.size < EXPECTED_MODEL_TASK_BYTES) {
|
||||
return invalidStatus(
|
||||
`Partial download in OPFS (${formatBytes(file.size)} of ${formatBytes(EXPECTED_MODEL_TASK_BYTES)}). Will resume on next install.`,
|
||||
manifest,
|
||||
file,
|
||||
);
|
||||
}
|
||||
const validationError = await validateTaskCandidate(file);
|
||||
if (validationError) {
|
||||
return invalidStatus(validationError, manifest, file);
|
||||
|
||||
@@ -30,7 +30,9 @@ export interface DirectChatTurnResult {
|
||||
|
||||
export type ClinicalChatStreamEvent =
|
||||
| AgentEvent
|
||||
| { type: 'thought_token'; partial: string };
|
||||
| { type: 'thought_token'; partial: string }
|
||||
/** A new local-model generation segment started (continuation) → spawn a new bubble. */
|
||||
| { type: 'segment_boundary'; segment: number };
|
||||
|
||||
export interface RunClinicalChatTurnInput {
|
||||
inferenceMode: InferenceMode;
|
||||
@@ -56,10 +58,17 @@ function buildChatHistory(
|
||||
if (message.streaming || !message.content.trim()) {
|
||||
continue;
|
||||
}
|
||||
if (message.role === 'user') {
|
||||
turns.push({ role: 'user', text: message.content });
|
||||
} else if (message.role === 'assistant') {
|
||||
turns.push({ role: 'assistant', text: message.content });
|
||||
if (message.role !== 'user' && message.role !== 'assistant') {
|
||||
continue;
|
||||
}
|
||||
const role: GemmaHistoryTurn['role'] = message.role === 'user' ? 'user' : 'assistant';
|
||||
// Continuation segments render as separate assistant bubbles; coalesce consecutive
|
||||
// same-role turns so Gemma sees one well-formed alternating turn.
|
||||
const previous = turns[turns.length - 1];
|
||||
if (previous && previous.role === role) {
|
||||
previous.text = `${previous.text}${message.content}`;
|
||||
} else {
|
||||
turns.push({ role, text: message.content });
|
||||
}
|
||||
}
|
||||
return turns.slice(-maxHistoryTurns * 2);
|
||||
@@ -126,6 +135,7 @@ export async function runDirectChatTurn(
|
||||
historyMessages: ClinicalChatMessage[];
|
||||
onToken?: (partial: string) => void;
|
||||
onThoughtToken?: (partial: string) => void;
|
||||
onSegmentStart?: (segment: number) => void;
|
||||
},
|
||||
signal?: AbortSignal,
|
||||
): Promise<DirectChatTurnResult> {
|
||||
@@ -206,6 +216,7 @@ export async function runDirectChatTurn(
|
||||
promptOptions,
|
||||
decode,
|
||||
input.onToken,
|
||||
input.onSegmentStart,
|
||||
);
|
||||
|
||||
if (signal?.aborted) {
|
||||
@@ -252,6 +263,7 @@ export async function runClinicalChatTurn(
|
||||
historyMessages: input.historyMessages,
|
||||
onToken: (partial) => onEvent?.({ type: 'final_token', partial }),
|
||||
onThoughtToken: (partial) => onEvent?.({ type: 'thought_token', partial }),
|
||||
onSegmentStart: (segment) => onEvent?.({ type: 'segment_boundary', segment }),
|
||||
},
|
||||
signal,
|
||||
);
|
||||
|
||||
@@ -227,6 +227,7 @@ function ClinicalWorkspaceContent({
|
||||
onSegmentationLoadingChange={setIsSegmentationLoading}
|
||||
patientMrn={patient.mrn}
|
||||
patientId={patient.id}
|
||||
useCelery={import.meta.env.VITE_USE_CV_CELERY !== 'false'}
|
||||
onRegisterSnapshotCapture={(capture) => {
|
||||
captureSnapshotRef.current = capture;
|
||||
}}
|
||||
|
||||
@@ -32,7 +32,9 @@ interface ImportMetaEnv {
|
||||
readonly VITE_OLLAMA_CHAT_URL?: string;
|
||||
readonly VITE_OLLAMA_MODEL?: string;
|
||||
readonly VITE_USE_BACKEND_SEGMENTATION?: string;
|
||||
readonly VITE_USE_CV_CELERY?: string;
|
||||
readonly VITE_SEGMENT_API_BASE?: string;
|
||||
readonly VITE_MODAL_OLLAMA_TARGET?: string;
|
||||
}
|
||||
|
||||
interface ImportMeta {
|
||||
|
||||
@@ -338,16 +338,16 @@ async function generateResponse(
|
||||
}
|
||||
|
||||
const baseBeforeSegment = combinedOutput;
|
||||
let emittedLength = combinedOutput.length;
|
||||
const tokenBatcher = createTokenBatcher(requestId, segmentNumber);
|
||||
let emittedSegmentLength = 0;
|
||||
|
||||
// Stream each segment's RAW tokens (its own thought channel + answer)
|
||||
// independently. The client renders one bubble per segment and parses its own
|
||||
// channel, so continuation thinking never leaks into a prior answer. The
|
||||
// cross-segment merge below is only for the model's continuation prompt + stats.
|
||||
const segmentRaw = await streamSegment(prompt, requestId, (_partial, segmentSoFar) => {
|
||||
const merged =
|
||||
segments === 0
|
||||
? segmentSoFar
|
||||
: mergeContinuationOutput(baseBeforeSegment, segmentSoFar, chainOfThought);
|
||||
const delta = merged.slice(emittedLength);
|
||||
emittedLength = merged.length;
|
||||
const delta = segmentSoFar.slice(emittedSegmentLength);
|
||||
emittedSegmentLength = segmentSoFar.length;
|
||||
if (delta.length > 0) {
|
||||
tokenBatcher.push(delta);
|
||||
}
|
||||
|
||||
@@ -15,7 +15,8 @@
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true,
|
||||
"noFallthroughCasesInSwitch": true,
|
||||
"noUncheckedSideEffectImports": true
|
||||
"noUncheckedSideEffectImports": true,
|
||||
"types": ["node"]
|
||||
},
|
||||
"include": ["src"]
|
||||
"include": ["src", "vite.config.ts"]
|
||||
}
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1,8 +1,27 @@
|
||||
import { defineConfig } from 'vite';
|
||||
import react from '@vitejs/plugin-react';
|
||||
import path from 'node:path';
|
||||
import fs from 'fs';
|
||||
import { load } from 'js-yaml';
|
||||
|
||||
function loadFrontendConfig(): Record<string, string> {
|
||||
try {
|
||||
const raw = fs.readFileSync('config/frontend.config.yaml', 'utf8');
|
||||
return load(raw) as Record<string, string>;
|
||||
} catch {
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
const frontendConfig = loadFrontendConfig();
|
||||
|
||||
const defineVars: Record<string, string> = {};
|
||||
for (const [key, value] of Object.entries(frontendConfig)) {
|
||||
defineVars[`import.meta.env.${key}`] = JSON.stringify(String(value));
|
||||
}
|
||||
|
||||
const MODAL_OLLAMA_TARGET =
|
||||
frontendConfig.VITE_MODAL_OLLAMA_TARGET ??
|
||||
'https://dtj-tran--ollama-gemma4-e4b-ollamaserver-web.modal.run';
|
||||
|
||||
export default defineConfig({
|
||||
@@ -52,4 +71,5 @@ export default defineConfig({
|
||||
},
|
||||
},
|
||||
},
|
||||
define: defineVars,
|
||||
});
|
||||
|
||||
@@ -1,31 +0,0 @@
|
||||
# Install Docker image
|
||||
docker network create jenkins
|
||||
|
||||
# install docker-integratable image
|
||||
docker run --name jenkins-docker --rm --detach \
|
||||
-p 8080:8080 -p 50000:50000 \
|
||||
--restart=on-failure \
|
||||
--privileged --network jenkins --network-alias docker \
|
||||
--env DOCKER_TLS_CERTDIR=/certs \
|
||||
--volume jenkins-docker-certs:/certs/client \
|
||||
--volume jenkins-data:/var/jenkins_home \
|
||||
--publish 2376:2376 \
|
||||
docker:dind --storage-driver overlay2 \
|
||||
-v jenkins_home:/var/jenkins_home jenkins/jenkins:lts
|
||||
|
||||
# install the docker images
|
||||
docker run \
|
||||
--name jenkins-blueocean \
|
||||
--restart=on-failure \
|
||||
--detach \
|
||||
--network jenkins \
|
||||
--env DOCKER_HOST=tcp://docker:2376 \
|
||||
--env DOCKER_CERT_PATH=/certs/client \
|
||||
--env DOCKER_TLS_VERIFY=1 \
|
||||
--publish 8080:8080 \
|
||||
--publish 50000:50000 \
|
||||
--volume jenkins-data:/var/jenkins_home \
|
||||
--volume jenkins-docker-certs:/certs/client:ro \
|
||||
jenkins/jenkins:lts
|
||||
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
|
||||
### 1. The Core Application Stack (Replacing the streaming engine)
|
||||
|
||||
* **Database:** **PostgreSQL** or **SQLite**
|
||||
* *Why:* Extremely lightweight, universally supported, and standard for medical/relational data (like patient exercises, joint ranges of motion, and logs).
|
||||
|
||||
|
||||
* **Backend:** **Node.js (Express)**, **Python (FastAPI)**, or **Go**
|
||||
* *Why:* Fast to build for a PoC, easily containerized via Docker, and highly efficient. FastAPI is excellent if you plan to incorporate any musculoskeletal data analysis or ML later.
|
||||
|
||||
|
||||
|
||||
### 2. CI/CD & Infrastructure Components
|
||||
|
||||
* **Codebase:** **Gitea**
|
||||
* *Why:* A lightweight, self-hosted Git platform that runs in a simple local Docker container using very little memory.
|
||||
|
||||
|
||||
* **CI/CD Orchestrator & Build Server:** **Woodpecker CI** or **GitHub Actions** (if you're okay using GitHub until migrating to a private cloud).
|
||||
* *Why:* Woodpecker is an open-source, container-first CI engine that runs locally with almost zero overhead.
|
||||
|
||||
|
||||
* **Artifact Repository:** **Gitea Packages** or **Docker Registry**
|
||||
* *Why:* You can store your built application images right inside Gitea or a tiny local Docker registry container.
|
||||
|
||||
|
||||
|
||||
### 3. Environments & Deployment
|
||||
|
||||
* **Deployment-Manager:** **Docker Compose** or **Portainer**
|
||||
* *Why:* To deploy your musculoskeletal app, you just need Docker Compose to orchestrate your backend and database containers. Portainer gives you a simple web GUI to manage them locally.
|
||||
|
||||
|
||||
* **Staging & Production Env:** Isolated local containers or separate virtual machines.
|
||||
|
||||
### 4. Feedback & Monitoring Loop
|
||||
|
||||
* **User Feedback Collector:** A custom-built form widget inside your app or an open-source tool like **Feedback Fish** / **Air Table**
|
||||
* **Feedback-Resolve:** **Focalboard** or **Leantime** (Self-hosted, lightweight project boards to track bugs and user feature requests).
|
||||
* **Monitoring & Logging:** **Prometheus + Grafana**
|
||||
* *Why:* Perfect for tracking application uptime, API response times, and server health.
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
|
||||
```planUML
|
||||
@startuml C4_Elements
|
||||
!include https://raw.githubusercontent.com/plantuml-stdlib/C4-PlantUML/master/C4_Component.puml
|
||||
|
||||
|
||||
title Component Diagram for CI/CD Pipeline Architecture
|
||||
|
||||
Person(developer, "Developer(s)", "Writes code and pushes changes.")
|
||||
Person(enduser, "End-User(s)", "Interacts with the production application.")
|
||||
|
||||
Container_Boundary(pipeline_boundary, "CI/CD Pipeline System") {
|
||||
Component(codebase, "Codebase", "Git Repository", "Stores the source code and tracks version history.")
|
||||
Component(orchestrator, "CI/CD Orchestrator", "Workflow Engine", "Triggers actions based on repository events.")
|
||||
Component(build_server, "Build Server", "Compiler/Packager", "Compiles code and builds release packages.")
|
||||
Component(artifact_repo, "Artifact Repository", "Storage", "Stores compiled binaries or container images.")
|
||||
Component(test_pipeline, "Test-Pipeline", "Automation Suite", "Runs unit, integration, and security tests.")
|
||||
Component(deploy_manager, "Deployment-Manager", "CD Engine", "Orchestrates deployment to various environments.")
|
||||
Component(user_feedback, "User Feedback Collector", "In-App/Portal Widget", "Collects explicit feature requests and bug reports from users.")
|
||||
Component(feedback_resolve, "Feedback-Resolve", "Tracking System", "Manages issues, bugs, and deployment logs.")
|
||||
Component(monitoring, "Monitoring & Logging", "Observability", "Monitors application health and metrics.")
|
||||
|
||||
Container_Boundary(deploy_env, "Deployment Environments") {
|
||||
Component(staging_env, "Staging/QA Env", "Environment", "Pre-production environment for testing.")
|
||||
Component(prod_env, "Production Env", "Environment", "Live environment hosting the customer application.")
|
||||
}
|
||||
}
|
||||
|
||||
' Directional Flow Links
|
||||
Rel(developer, codebase, "1. Pushes code")
|
||||
Rel(codebase, orchestrator, "2. Triggers event webhook")
|
||||
Rel(orchestrator, build_server, "3. Triggers build job")
|
||||
Rel(build_server, artifact_repo, "4. Stores compiled artifact")
|
||||
Rel(orchestrator, test_pipeline, "5. Runs automated tests")
|
||||
Rel(orchestrator, deploy_manager, "6. Signals ready for deployment")
|
||||
|
||||
Rel(deploy_manager, artifact_repo, "7. Pulls latest artifact")
|
||||
Rel(deploy_manager, staging_env, "8. Deploys to")
|
||||
Rel(deploy_manager, prod_env, "9. Promotes approved changes to")
|
||||
|
||||
Rel(enduser, prod_env, "10. Interacts with application")
|
||||
Rel(enduser, user_feedback, "11. Submits requests & feedback")
|
||||
Rel(user_feedback, feedback_resolve, "12. Forwards user tickets")
|
||||
Rel(monitoring, prod_env, "13. Reads metrics and errors")
|
||||
Rel(monitoring, feedback_resolve, "14. Feeds performance/error logs")
|
||||
Rel(feedback_resolve, developer, "15. Alerts for loop closure")
|
||||
|
||||
@enduml
|
||||
```
|
||||
@@ -205,6 +205,9 @@ async def forward_list_models():
|
||||
min_containers=1, # for keeping warm and prevention,
|
||||
buffer_containers=2, # Number of additional idle containers to maintain under active load.
|
||||
scaledown_window=30, # Max time (in seconds) a container can remain idle while scaling down.
|
||||
volumes= {
|
||||
'/mnt/vkist-ml-model' : modal.CloudBucketMount(bucket_name="vkist-ml-model", secret=modal.Secret.from_name("aws-secrets"))
|
||||
},
|
||||
secrets=[modal.Secret.from_name("aws-secrets")]
|
||||
)
|
||||
@modal.asgi_app()
|
||||
@@ -213,7 +216,11 @@ def unified_triton_server():
|
||||
|
||||
# Spawns Triton in the background. It will automatically read
|
||||
# your "aws-secrets" environment keys to mount s3://vkist-ml-model/
|
||||
cmd = ["tritonserver", "--model-repository=s3://vkist-ml-model/"]
|
||||
# cmd = ["tritonserver", "--model-repository=s3://vkist-ml-model/"] # bad pattern causing latency
|
||||
cmd = ["tritonserver", "--model-repository=/mnt/vkist-ml-model/"]
|
||||
|
||||
# triton issue network connection -> AWS -> adding cold-off latency
|
||||
# idea mounting the model to Modal Volume
|
||||
subprocess.Popen(cmd)
|
||||
|
||||
print("📋 Triton background process delegated. Handing routing control over to FastAPI.")
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
cd ../PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/infra/implementation/triton_run
|
||||
MODAL_PROFILE=dtj-tran modal deploy PILOT_PROJECT/workspace/sprint_1_2/codebase/infra/implementation/triton_run/modal_triton.py
|
||||
MODAL_PROFILE=dtj-tran modal deploy modal_triton.py
|
||||
@@ -95,8 +95,9 @@ vibe-reader==0.2.1
|
||||
work-with-database==0.0.1
|
||||
x-lib==0.0.27
|
||||
# WARNING(pigar): the following duplicate requirements are for the import name: langchain_google_vertexai
|
||||
gigachain-google-vertexai==2.0.0
|
||||
langchain-google-vertexai==3.2.4
|
||||
# WARNING(pigar): the following duplicate requirements are for the import name: optimum
|
||||
optimum==2.1.0
|
||||
optimum-onnx==0.1.0
|
||||
Celery==5.6.3
|
||||
redis==8.0.1
|
||||
BIN
workspace/sprint_1_2/PRODUCT_VISUALIZATION/home_screen.png
Normal file
BIN
workspace/sprint_1_2/PRODUCT_VISUALIZATION/home_screen.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 722 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.1 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.3 MiB |
Reference in New Issue
Block a user