msk_admin/Lumina-MSK
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

test_workflow
Some checks failed
Backend ECR Deployment / build-and-push (push) Has been cancelled
Details
Backend ECR Deployment / notify-deploy (push) Has been cancelled
Details
Backend Modal Build / notify-deploy (push) Has been cancelled
Details
Backend Modal Build / Build & Push via Modal (push) Has been cancelled
Details

Browse Source
This commit is contained in:
DatTT127
2026-07-18 17:48:19 +07:00
parent 7d5c583475
commit 57a8bac1be
87 changed files with 36291 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files

118
workspace/sprint_1_2/CODEBASE/deps/implementation/backend_deploy/Dockerfile Normal file
View File

@@ -0,0 +1,118 @@
# =============================================================================
# Stage 1: Wheelhouse Builder
# Pre-compiles all Python dependencies into .whl files into /wheels.
# This layer is cached independently and only rebuilds when requirements.txt
# changes. Heavy packages (torch, opencv, pycocotools, etc.) are compiled once
# per platform (amd64/arm64) and reused by all subsequent builds.
# =============================================================================
FROM python:3.12-slim-bookworm AS wheelhouse
WORKDIR /wheels
# Install build tools required for packages with C extensions
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
g++ \
make \
libgl1 \
libglib2.0-0 \
libxcb1 \
libx11-6 \
libxext6 \
libsm6 \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements first so this layer is cached unless deps change
COPY requirements.txt .
# Build wheel files for all dependencies AND their transitive dependencies.
# --wheel-dir=/wheels: write .whl files to the wheelhouse directory
# (no --no-deps: we want the full dependency tree pre-compiled)
RUN pip wheel --no-cache-dir --wheel-dir=/wheels -r requirements.txt
# =============================================================================
# Stage 2: Application Builder
# Creates a venv and installs dependencies from the local wheelhouse.
# Because pip installs from /wheels (not PyPI), this layer stays cached as
# long as the wheelhouse is unchanged — application code changes do NOT
# trigger re-downloading or recompiling any dependency.
# =============================================================================
FROM python:3.12-slim-bookworm AS builder
WORKDIR /app
# Install build tools (kept for safety; wheels are pre-compiled but some
# packages may still invoke build steps during install)
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
g++ \
make \
libgl1 \
libglib2.0-0 \
libxcb1 \
libx11-6 \
libxext6 \
libsm6 \
&& rm -rf /var/lib/apt/lists/*
# Create virtual environment so we can copy a single tree to runtime
RUN python -m venv /opt/venv
ENV PATH=/opt/venv/bin:$PATH
# Copy requirements for pip install validation
COPY requirements.txt .
# Copy pre-built wheels from wheelhouse stage (platform-specific)
COPY --from=wheelhouse /wheels /wheels
# Install dependencies FROM LOCAL WHEELS ONLY (no network access needed)
# --no-index: do not query PyPI or any external index
# --find-links: use only the local wheelhouse at /wheels
RUN pip install --no-cache-dir --no-index --find-links=/wheels -r requirements.txt
# =============================================================================
# Stage 3: Runtime
# Minimal image containing only the venv and application code.
# =============================================================================
FROM python:3.12-slim-bookworm AS runtime
WORKDIR /app
# Create non-root user
# RUN groupadd -r appgroup && useradd -r -g appgroup -d /app -s /sbin/nologin appuser
# # Copy the entire venv from builder
# COPY --from=builder --chown=appuser:appgroup /opt/venv /opt/venv
# Step 2: Assign an explicit numeric ID (like 999) when creating the user/group
RUN groupadd -g 999 appgroup && \
useradd -r -u 999 -g appgroup -d /app -s /sbin/nologin appuser
RUN apt-get update && apt-get install -y --no-install-recommends \
libgl1 \
libglib2.0-0 \
libxcb1 \
libx11-6 \
libxext6 \
libsm6 \
&& rm -rf /var/lib/apt/lists/*
# Step 3: Use the exact matching numeric IDs for --chown
COPY --from=builder --chown=999:999 /opt/venv /opt/venv
ENV PATH=/opt/venv/bin:$PATH
# Copy application code
COPY --chown=999:999 backend ./backend
# Switch to non-root user
USER appuser
# Expose port
EXPOSE 8001
# Health check
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8001/health')" || exit 1
# Run the CV inference server
ENTRYPOINT ["python", "-m", "backend.cv_inference_server"]

22
workspace/sprint_1_2/CODEBASE/deps/implementation/backend_deploy/docker-compose.backend.yaml Normal file
View File

@@ -0,0 +1,22 @@
version: '3.8'
services:
cv-inference-server:
image: public.ecr.aws/${ECR_PUBLIC_REGISTRY_ALIAS}/msk-cv-inference-server:latest
container_name: cv-inference-server
restart: unless-stopped
ports:
- "8001:8001"
# Environment variables passed from shell (injected by webhook listener from Gitea vars)
# No .env file needed — all config comes from Gitea repo variables via webhook payload
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8001/health')"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
networks:
default:
name: pilot-network
external: true

328
workspace/sprint_1_2/CODEBASE/deps/implementation/backend_deploy/gitea_modal_build.py Normal file
View File

@@ -0,0 +1,328 @@
"""
Modal script to build and push VKIST backend Docker image to container registry.
Run from Gitea runner on Lightsail (2GB RAM) - offloads heavy build to Modal serverless.
Usage:
modal run gitea_modal_build.py \
--registry public.ecr.aws/vkist-project \
--image-name vkist-backend \
--tag v1.2.3 \
--push
# Or deploy as reusable Modal app:
modal deploy gitea_modal_build.py
"""
import os
from pathlib import Path
import modal
# =============================================================================
# Configuration - Dynamic paths relative to this script
# =============================================================================
# Get the directory where this script actually lives on the machine running it
SCRIPT_DIR = Path(__file__).resolve().parent
# Automatically resolve the root of the repository (climbing up 3 levels)
# From: deps/implementation/backend_deploy/gitea_modal_build.py
# To: CODEBASE/
PROJECT_ROOT = SCRIPT_DIR.parents[2]
# Explicitly find your local Dockerfile
DOCKERFILE_PATH = SCRIPT_DIR / "Dockerfile"
print(f"--- Environment Verification ---")
print(f"Script location: {SCRIPT_DIR}")
print(f"Resolved PROJECT_ROOT: {PROJECT_ROOT} (Exists: {PROJECT_ROOT.exists()})")
print(f"Resolved DOCKERFILE_PATH: {DOCKERFILE_PATH} (Exists: {DOCKERFILE_PATH.exists()})")
print(f"----------------------------------------")
# Project layout variables
BACKEND_SOURCE = "backend"
REQUIREMENTS_FILE = "requirements.txt"
# Default registry/image settings
DEFAULT_REGISTRY = "public.ecr.aws/i9a4e3f6/msk-cv-inference-server"
DEFAULT_IMAGE_NAME = "msk-lumina-backend"
DEFAULT_TAG = "latest"
# Modal app configuration
APP_NAME = "msk-lumina-backend-builder"
# =============================================================================
# Modal Image Definition - uses the project's Dockerfile for consistency
# =============================================================================
# Build the image using the existing multi-stage Dockerfile
# We set context_dir to PROJECT_ROOT so the Dockerfile can see ../../../../backend
backend_image = modal.Image.from_dockerfile(
str(DOCKERFILE_PATH),
context_dir=str(PROJECT_ROOT),
).pip_install(
"docker", # for test_image function
"boto3", # for ECR auth if needed
)
app = modal.App(APP_NAME, image=backend_image)
# =============================================================================
# Build and Push Function
# =============================================================================
@app.function(
timeout=900, # 15 min for heavy ML deps (torch, transformers, etc.)
cpu=4,
memory=8192, # 8GB RAM - plenty for wheel building
secrets=[
modal.Secret.from_name("aws-secrets"), # AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION
# modal.Secret.from_name("dockerhub-credentials"), # DOCKERHUB_USERNAME, DOCKERHUB_TOKEN (optional)
# modal.Secret.from_name("gcp-artifact-registry"), # GCP credentials (optional)
],
)
def build_and_push(
registry: str = DEFAULT_REGISTRY,
image_name: str = DEFAULT_IMAGE_NAME,
tag: str = DEFAULT_TAG,
push: bool = True,
platform: str = "linux/amd64",
) -> dict:
"""
Build the backend Docker image and push to registry.
Args:
registry: Container registry hostname (e.g., public.ecr.aws/vkist-project)
image_name: Image name (e.g., vkist-backend)
tag: Image tag (e.g., v1.2.3, latest, git-sha)
push: Whether to push to registry
platform: Target platform (linux/amd64, linux/arm64)
Returns:
Dict with image reference, digest, and size info
"""
import docker
full_image = f"{registry}/{image_name}:{tag}"
print(f"Building {full_image} for {platform}")
# Initialize Docker client (uses Modal's container runtime)
client = docker.from_env()
# Build using the Dockerfile from project root context
print(f"Build context: {PROJECT_ROOT}")
print(f"Dockerfile: {DOCKERFILE_PATH}")
# Build the image
image, build_logs = client.images.build(
path=PROJECT_ROOT,
dockerfile=DOCKERFILE_PATH,
tag=full_image,
platform=platform,
rm=True,
pull=True,
buildargs={
"BUILDKIT_INLINE_CACHE": "1",
},
)
# Stream build logs
for chunk in build_logs:
if "stream" in chunk:
print(chunk["stream"].strip())
elif "error" in chunk:
raise RuntimeError(f"Build failed: {chunk['error']}")
print(f"Build complete: {image.id[:12]}")
# Get image details
image_info = client.images.get(full_image)
size_bytes = image_info.attrs["Size"]
digest = image_info.attrs.get("RepoDigests", ["unknown"])[0]
result = {
"image": full_image,
"digest": digest,
"size_bytes": size_bytes,
"image_id": image.id,
}
if push:
print(f"Pushing to {registry}...")
# Authenticate with registry based on hostname
_authenticate_registry(registry, client)
# Push image
push_logs = client.images.push(
repository=f"{registry}/{image_name}",
tag=tag,
stream=True,
decode=True,
)
for chunk in push_logs:
if "status" in chunk:
print(f" {chunk['status']}: {chunk.get('progress', '')}")
if "error" in chunk:
raise RuntimeError(f"Push failed: {chunk['error']}")
print(f"Push complete: {full_image}")
# Get digest after push
pushed_image = client.images.get(full_image)
result["digest"] = pushed_image.attrs.get("RepoDigests", ["unknown"])[0]
return result
def _authenticate_registry(registry: str, client):
"""Authenticate Docker client with the target registry."""
import subprocess
if "ecr.aws" in registry or "ecr." in registry:
# AWS ECR (public or private)
region = os.getenv("AWS_REGION", "us-east-1")
print(f"Authenticating with AWS ECR ({region})...")
subprocess.run([
"aws", "ecr", "get-login-password", "--region", region
], check=True, capture_output=True)
# Note: Modal's aws-ecr-credentials secret should handle this via env vars
elif "docker.io" in registry or "index.docker.io" in registry or "/" not in registry.split("/")[0]:
# Docker Hub
username = os.getenv("DOCKERHUB_USERNAME")
password = os.getenv("DOCKERHUB_TOKEN")
if username and password:
print("Authenticating with Docker Hub...")
client.login(username=username, password=password)
elif "gcr.io" in registry or "pkg.dev" in registry:
# Google Container Registry / Artifact Registry
print("Authenticating with GCP Artifact Registry...")
subprocess.run(["gcloud", "auth", "configure-docker", "--quiet"], check=True)
@app.function(
image=backend_image,
timeout=120,
secrets=[modal.Secret.from_name("aws-secrets")],
)
def test_image(registry: str, image_name: str, tag: str) -> dict:
"""Test the built image by running a quick health check."""
import docker
full_image = f"{registry}/{image_name}:{tag}"
client = docker.from_env()
print(f"Pulling {full_image} for testing...")
client.images.pull(full_image)
# print("Running health check...")
# output = client.containers.run(
# full_image,
# command=["python", "-c", "import backend.cv_inference_server; print('Import OK')"],
# remove=True,
# detach=False,
# )
print("Running basic smoke test...")
output = client.containers.run(
full_image,
command=[
"python", "-c",
"import sys; print('Python', sys.version); "
"import cv2; print('OpenCV', cv2.__version__); "
# "import backend.cv_inference_server; print('Import OK')",
],
remove=True,
detach=False,
)
return {
"status": "healthy",
"output": output.decode().strip(),
"image": full_image,
}
# =============================================================================
# CLI Entry Point
# =============================================================================
@app.local_entrypoint()
def main(
registry: str = DEFAULT_REGISTRY,
image_name: str = DEFAULT_IMAGE_NAME,
tag: str = DEFAULT_TAG,
push: bool = True,
platform: str = "linux/amd64",
test: bool = False,
):
"""
Build and push backend Docker image via Modal.
Args:
registry: Container registry (default: public.ecr.aws/vkist-project)
image_name: Image name (default: vkist-backend)
tag: Image tag (default: latest)
push: Push to registry (default: true)
platform: Target platform (default: linux/amd64)
test: Run health check after build (default: false)
"""
print("=" * 60)
print("VKIST Backend Docker Build on Modal")
print("=" * 60)
print(f"Registry: {registry}")
print(f"Image: {image_name}")
print(f"Tag: {tag}")
print(f"Push: {push}")
print(f"Platform: {platform}")
print(f"Test: {test}")
print("=" * 60)
# Build and push
result = build_and_push.remote(
registry=registry,
image_name=image_name,
tag=tag,
push=push,
platform=platform,
)
print("\n" + "=" * 60)
print("BUILD COMPLETE")
print("=" * 60)
print(f"Image: {result['image']}")
print(f"Digest: {result['digest']}")
print(f"Size: {result['size_bytes'] / 1024 / 1024:.1f} MB")
print("=" * 60)
# Output for CI/CD parsing (GitHub Actions / Gitea Actions format)
print(f"::set-output name=image::{result['image']}")
print(f"::set-output name=digest::{result['digest']}")
print(f"::set-output name=size_mb::{result['size_bytes'] / 1024 / 1024:.1f}")
# Optional test
if test and push:
print("\nRunning health check...")
test_result = test_image.remote(registry, image_name, tag)
print(f"Health check: {test_result['status']}")
print(f"Output: {test_result['output']}")
if __name__ == "__main__":
# Allow direct execution for debugging
import sys
if len(sys.argv) > 1 and sys.argv[1] in ("-h", "--help"):
print(__doc__)
print("\nExamples:")
print(" modal run gitea_modal_build.py --tag v1.0.0")
print(" modal run gitea_modal_build.py --registry docker.io/myuser --image-name myapp --tag latest")
print(" modal deploy gitea_modal_build.py")
else:
print("Use: modal run gitea_modal_build.py [options]")
print("Or: modal deploy gitea_modal_build.py")

110
workspace/sprint_1_2/CODEBASE/deps/implementation/backend_deploy/modal_build.sh Executable file
View File

@@ -0,0 +1,110 @@
#!/usr/bin/env bash
#
# Wrapper script for Gitea runner to invoke Modal build.
# Runs on Lightsail VM (2GB RAM) - offloads Docker build to Modal serverless.
#
# Usage: ./modal_build.sh [TAG] [REGISTRY] [IMAGE_NAME] [PLATFORM]
# TAG - Image tag (default: git SHA or 'latest')
# REGISTRY - Container registry (default: public.ecr.aws/vkist-project)
# IMAGE_NAME - Image name (default: vkist-backend)
# PLATFORM - Target platform (default: linux/amd64)
#
# Environment variables (set as Gitea repository secrets):
# MODAL_TOKEN - Modal API token (from modal.com/settings)
# AWS_ACCESS_KEY_ID - For ECR push
# AWS_SECRET_ACCESS_KEY
# AWS_REGION - Default: us-east-1
# DOCKERHUB_USERNAME - Optional, for Docker Hub
# DOCKERHUB_TOKEN - Optional, for Docker Hub
#
# Example:
# ./modal_build.sh v1.2.3
# ./modal_build.sh latest public.ecr.aws/my-project my-app linux/arm64
set -euo pipefail
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODAL_SCRIPT="${SCRIPT_DIR}/gitea_modal_build.py"
# Defaults
DEFAULT_REGISTRY="public.ecr.aws/vkist-project"
DEFAULT_IMAGE_NAME="vkist-backend"
DEFAULT_PLATFORM="linux/amd64"
# Parse arguments
TAG="${1:-${GITEA_SHA:-latest}}"
REGISTRY="${2:-${DEFAULT_REGISTRY}}"
IMAGE_NAME="${3:-${DEFAULT_IMAGE_NAME}}"
PLATFORM="${4:-${DEFAULT_PLATFORM}}"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
log_info() { echo -e "${GREEN}[INFO]${NC} $*"; }
log_warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
log_error() { echo -e "${RED}[ERROR]${NC} $*"; }
# Check prerequisites
check_prereqs() {
log_info "Checking prerequisites..."
if ! command -v modal &> /dev/null; then
log_error "Modal CLI not found. Install with: pip install modal"
exit 1
fi
if [ ! -f "$MODAL_SCRIPT" ]; then
log_error "Modal script not found at: $MODAL_SCRIPT"
exit 1
fi
# Check Modal auth
if [ -z "${MODAL_TOKEN:-}" ]; then
log_warn "MODAL_TOKEN not set. Ensure 'modal setup' was run or token is in environment."
fi
log_info "Prerequisites OK"
}
# Build via Modal
build_via_modal() {
log_info "Starting Modal build..."
log_info "Registry: $REGISTRY"
log_info "Image: $IMAGE_NAME"
log_info "Tag: $TAG"
log_info "Platform: $PLATFORM"
# Run modal build
modal run "$MODAL_SCRIPT" \
--registry "$REGISTRY" \
--image-name "$IMAGE_NAME" \
--tag "$TAG" \
--platform "$PLATFORM" \
--push
local exit_code=$?
if [ $exit_code -eq 0 ]; then
log_info "Build and push completed successfully!"
log_info "Image: $REGISTRY/$IMAGE_NAME:$TAG"
else
log_error "Modal build failed with exit code $exit_code"
exit $exit_code
fi
}
# Main
main() {
log_info "=== VKIST Modal Docker Build Wrapper ==="
check_prereqs
build_via_modal
log_info "=== Done ==="
}
main "$@"

64
workspace/sprint_1_2/CODEBASE/deps/implementation/backend_deploy/webhook-deploy-backend.sh Executable file
View File

@@ -0,0 +1,64 @@
#!/bin/bash
# Deploy script for VKIST CV Inference Server (Backend)
# Triggered by Gitea webhook — env vars come from webhook payload, NOT .env file
set -euo pipefail
# Configuration
PROJECT_DIR="/opt/pilot-project"
COMPOSE_FILE="${PROJECT_DIR}/docker-compose.backend.yaml"
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*"
}
main() {
log "Starting CV inference server deployment"
# Verify compose file exists
if [[ ! -f "${COMPOSE_FILE}" ]]; then
log "ERROR: Compose file not found at ${COMPOSE_FILE}"
exit 1
fi
cd "${PROJECT_DIR}"
# Pull latest compose file (in case it was updated)
log "Pulling latest compose file from git"
git pull --quiet origin main || log "WARNING: git pull failed, using local compose file"
# Pull latest image
# ECR_PUBLIC_REGISTRY_ALIAS comes from webhook env payload
log "Pulling latest image: public.ecr.aws/${ECR_PUBLIC_REGISTRY_ALIAS}/msk-cv-inference-server:latest"
docker compose -f "${COMPOSE_FILE}" pull cv-inference-server
# Deploy with zero-downtime recreation
# All required env vars (TRITON_ENDPOINT, CV_INFERENCE_HOST, etc.) are already
# in the shell environment, passed from webhook payload
log "Recreating cv-inference-server container"
docker compose -f "${COMPOSE_FILE}" up -d --force-recreate cv-inference-server
# Clean up old images
log "Pruning unused images"
docker image prune -f
# Verify health
log "Waiting for health check..."
for i in {1..30}; do
if docker compose -f "${COMPOSE_FILE}" exec -T cv-inference-server \
python -c "import urllib.request; urllib.request.urlopen('http://localhost:8001/health')" 2>/dev/null; then
log "Health check passed"
break
fi
if [[ $i -eq 30 ]]; then
log "ERROR: Health check failed after 30 attempts"
docker compose -f "${COMPOSE_FILE}" logs cv-inference-server
exit 1
fi
sleep 2
done
log "Deployment completed successfully"
}
main "$@"

18
workspace/sprint_1_2/CODEBASE/deps/implementation/frontend_deploy/Caddyfile.example Normal file
View File

@@ -0,0 +1,18 @@
# /opt/pilot-project/Caddyfile
# Replace app.example.com with your actual domain
app.example.com {
reverse_proxy lumina-frontend:80
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "SAMEORIGIN"
Referrer-Policy "strict-origin-when-cross-origin"
}
@static {
path *.js *.css *.wasm *.png *.jpg *.svg *.woff2 *.ico *.map
}
header @static Cache-Control "public, max-age=31536000, immutable"
}

0
workspace/sprint_1_2/CODEBASE/deps/implementation/frontend_deploy/Dockerfile Normal file
View File

25
workspace/sprint_1_2/CODEBASE/deps/implementation/frontend_deploy/docker-compose.frontend.yaml Normal file
View File

@@ -0,0 +1,25 @@
version: '3.8'
services:
lumina-frontend:
image: public.ecr.aws/${ECR_PUBLIC_REGISTRY_ALIAS}/lumina-frontend:latest
container_name: lumina-frontend
restart: unless-stopped
ports:
- "8080:80"
deploy:
resources:
limits:
memory: 128m
cpus: '0.5'
healthcheck:
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost/health"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
networks:
default:
name: pilot-network
external: true

14
workspace/sprint_1_2/CODEBASE/deps/implementation/frontend_deploy/env.example Normal file
View File

@@ -0,0 +1,14 @@
# Lightsail Production Environment
# Copy to /opt/pilot-project/.env on Lightsail VM
# chmod 600 /opt/pilot-project/.env
# ECR Public Registry Alias (e.g., vkist-pilot, your-account-alias)
# Get from: AWS Console > ECR Public > Registries > [your registry] > Alias
ECR_PUBLIC_REGISTRY_ALIAS=vkist-pilot
# Webhook Secret (generate with: openssl rand -hex 32)
# Must match Gitea webhook secret configuration
WEBHOOK_SECRET=your-generated-secret-here
# Optional: Backend API URL for nginx proxy (if different from frontend)
# BACKEND_API_URL=http://localhost:8001

26
workspace/sprint_1_2/CODEBASE/deps/implementation/frontend_deploy/gitea-webhook-listener.service Normal file
View File

@@ -0,0 +1,26 @@
[Unit]
Description=Gitea Webhook Listener for Lumina Frontend Deploy
After=network.target docker.service
Requires=docker.service
[Service]
Type=simple
User=ubuntu
Group=ubuntu
WorkingDirectory=/opt/pilot-project
EnvironmentFile=/opt/pilot-project/.env
ExecStart=/usr/bin/python3 /opt/pilot-project/webhook-listener.py
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
# Security hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/opt/pilot-project
ProtectHome=true
[Install]
WantedBy=multi-user.target

82
workspace/sprint_1_2/CODEBASE/deps/implementation/frontend_deploy/webhook-deploy.sh Executable file
View File

@@ -0,0 +1,82 @@
#!/bin/bash
# Deploy script for Lumina MSK Frontend
# Triggered by Gitea webhook - receives env vars from Gitea repo variables
set -euo pipefail
# Configuration
PROJECT_DIR="/opt/pilot-project"
COMPOSE_FILE="${PROJECT_DIR}/docker-compose.frontend.yaml"
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*"
}
# Required env vars (passed from webhook / Gitea variables)
REQUIRED_VARS=(
"ECR_PUBLIC_REGISTRY_ALIAS"
"CORS_ORIGINS"
)
# Write .env from passed environment variables (so docker compose env_file works)
write_env_file() {
local env_file="${PROJECT_DIR}/.env"
log "Writing .env from passed environment variables"
for var in "${REQUIRED_VARS[@]}"; do
if [[ -n "${!var:-}" ]]; then
echo "${var}=${!var}" >> "${env_file}"
else
log "WARNING: ${var} not set in environment"
fi
done
}
main() {
log "Starting frontend deployment"
# Verify compose file exists
if [[ ! -f "${COMPOSE_FILE}" ]]; then
log "ERROR: Compose file not found at ${COMPOSE_FILE}"
exit 1
fi
cd "${PROJECT_DIR}"
# Pull latest compose file (in case it was updated)
log "Pulling latest compose file from git"
git pull --quiet origin main || log "WARNING: git pull failed, using local compose file"
# Write .env from passed env vars
write_env_file
# Pull latest image
log "Pulling latest image: public.ecr.aws/${ECR_PUBLIC_REGISTRY_ALIAS}/lumina-frontend:latest"
docker compose -f "${COMPOSE_FILE}" pull lumina-frontend
# Deploy with zero-downtime recreation
log "Recreating frontend container"
docker compose -f "${COMPOSE_FILE}" up -d --force-recreate lumina-frontend
# Clean up old images
log "Pruning unused images"
docker image prune -f
# Verify health
log "Waiting for health check..."
for i in {1..30}; do
if docker compose -f "${COMPOSE_FILE}" exec -T lumina-frontend wget --quiet --tries=1 --spider http://localhost/ 2>/dev/null; then
log "Health check passed"
break
fi
if [[ $i -eq 30 ]]; then
log "ERROR: Health check failed after 30 attempts"
docker compose -f "${COMPOSE_FILE}" logs lumina-frontend
exit 1
fi
sleep 2
done
log "Deployment completed successfully"
}
main "$@"

178
workspace/sprint_1_2/CODEBASE/deps/implementation/frontend_deploy/webhook-listener.py Executable file
View File

@@ -0,0 +1,178 @@
#!/usr/bin/env python3
"""
Gitea Webhook Listener for Lumina Deployment
Runs on Lightsail VM, receives webhook from Gitea, triggers the appropriate
deploy script based on which workflow completed.
"""
import os
import hmac
import hashlib
import subprocess
import logging
from http.server import HTTPServer, BaseHTTPRequestHandler
from urllib.parse import urlparse, parse_qs
# Configuration
WEBHOOK_SECRET = os.environ.get('WEBHOOK_SECRET')
DEPLOY_SCRIPTS = {
'Frontend CI/CD': '/opt/pilot-project/webhook-deploy.sh',
'Backend ECR Deployment': '/opt/pilot-project/webhook-deploy-backend.sh',
'Backend Modal Build': '/opt/pilot-project/webhook-deploy-backend.sh',
}
LISTEN_HOST = '127.0.0.1'
LISTEN_PORT = 3333
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
class WebhookHandler(BaseHTTPRequestHandler):
def do_POST(self):
# Verify path
if self.path != '/deploy':
self.send_response(404)
self.end_headers()
return
# Read body
content_length = int(self.headers.get('Content-Length', 0))
body = self.rfile.read(content_length)
# Verify signature
signature = self.headers.get('X-Gitea-Signature', '')
if not self.verify_signature(body, signature):
logger.warning("Invalid webhook signature")
self.send_response(401)
self.end_headers()
self.wfile.write(b'Invalid signature')
return
# Parse JSON payload
try:
import json
payload = json.loads(body.decode('utf-8'))
except json.JSONDecodeError:
logger.error("Invalid JSON payload")
self.send_response(400)
self.end_headers()
return
# Check if workflow completed successfully
deploy_info = self.should_deploy(payload)
if not deploy_info:
logger.info("Event does not trigger deploy: %s", payload.get('action', 'unknown'))
self.send_response(200)
self.end_headers()
self.wfile.write(b'Event ignored')
return
deploy_script = deploy_info['script']
deploy_env = deploy_info.get('env', {})
# Trigger deployment with env vars from payload
logger.info("Triggering deployment with script: %s", deploy_script)
logger.debug("Passing env vars: %s", list(deploy_env.keys()))
try:
# Merge payload env vars with current process env
proc_env = os.environ.copy()
proc_env.update(deploy_env)
result = subprocess.run(
[deploy_script],
capture_output=True,
text=True,
timeout=300,
env=proc_env
)
if result.returncode == 0:
logger.info("Deployment successful")
logger.debug("Deploy output: %s", result.stdout)
self.send_response(200)
self.end_headers()
self.wfile.write(b'Deployment triggered successfully')
else:
logger.error("Deployment failed: %s", result.stderr)
self.send_response(500)
self.end_headers()
self.wfile.write(f'Deployment failed: {result.stderr}'.encode())
except subprocess.TimeoutExpired:
logger.error("Deployment timed out")
self.send_response(500)
self.end_headers()
self.wfile.write(b'Deployment timed out')
except Exception as e:
logger.exception("Deployment error")
self.send_response(500)
self.end_headers()
self.wfile.write(f'Deployment error: {str(e)}'.encode())
def verify_signature(self, body: bytes, signature: str) -> bool:
"""Verify Gitea HMAC-SHA256 signature"""
if not WEBHOOK_SECRET:
logger.error("WEBHOOK_SECRET not configured")
return False
expected = 'sha256=' + hmac.new(
WEBHOOK_SECRET.encode(),
body,
hashlib.sha256
).hexdigest()
return hmac.compare_digest(expected, signature)
def should_deploy(self, payload: dict) -> dict | None:
"""Determine if this webhook event should trigger a deploy.
Returns dict with 'script' and optional 'env' if yes, None if no."""
# Gitea workflow_run event
if payload.get('action') == 'completed':
workflow = payload.get('workflow', {})
workflow_name = workflow.get('name', '')
conclusion = payload.get('conclusion')
if conclusion == 'success' and workflow_name in DEPLOY_SCRIPTS:
return {
'script': DEPLOY_SCRIPTS[workflow_name],
'env': payload.get('env', {})
}
# Direct webhook payload (for manual triggers or extended format)
if 'service' in payload:
service = payload.get('service')
if service == 'cv-inference-server':
return {
'script': DEPLOY_SCRIPTS['Backend ECR Deployment'],
'env': payload.get('env', {})
}
if service == 'lumina-frontend':
return {
'script': DEPLOY_SCRIPTS['Frontend CI/CD'],
'env': payload.get('env', {})
}
return None
def log_message(self, format, *args):
logger.info("%s - %s", self.address_string(), format % args)
def main():
if not WEBHOOK_SECRET:
logger.error("WEBHOOK_SECRET environment variable not set")
exit(1)
# Verify deploy scripts exist
for name, path in DEPLOY_SCRIPTS.items():
if not os.path.exists(path):
logger.warning("Deploy script not found for %s at %s", name, path)
server = HTTPServer((LISTEN_HOST, LISTEN_PORT), WebhookHandler)
logger.info("Webhook listener starting on %s:%s", LISTEN_HOST, LISTEN_PORT)
logger.info("Watching workflows: %s", ', '.join(DEPLOY_SCRIPTS.keys()))
try:
server.serve_forever()
except KeyboardInterrupt:
logger.info("Shutting down")
server.shutdown()
if __name__ == '__main__':
main()

0
workspace/sprint_1_2/CODEBASE/deps/implementation/Dockerfile → workspace/sprint_1_2/CODEBASE/deps/implementation/gitea_worker/Dockerfile
View File

342
workspace/sprint_1_2/CODEBASE/deps/spec/ci_cd_frontend/EC_PUBLIC_SETUP_GUIDE.md Normal file
View File

@@ -0,0 +1,342 @@
# ECR Public CI/CD Setup Guide (with Caddy Reverse Proxy)
Complete setup for Lumina MSK Frontend: Gitea CI → Amazon ECR Public → Lightsail Deploy with Caddy (auto-HTTPS)
---
## Prerequisites
| Requirement | Version/Details |
|-------------|-----------------|
| AWS Account | With ECR Public access |
| Lightsail VM | 2 GB, Ubuntu 22.04+, Docker + Docker Compose installed |
| Gitea Server | Running on Lightsail (or accessible), Actions enabled |
| Domain | For HTTPS via Let's Encrypt (Caddy handles this automatically) |
---
## 1. Amazon ECR Public Setup
### 1.1 Create Public Registry
```bash
# AWS Console → ECR Public → Create registry
# Note the Registry Alias (e.g., vkist-pilot)
# Region: us-east-1 (only region for ECR Public)
```
### 1.2 Create Repository
```bash
aws ecr-public create-repository \
--repository-name lumina-frontend \
--region us-east-1
```
### 1.3 Create IAM User for CI
```bash
aws iam create-user --user-name gitea-ci-ecr-public
```
### 1.4 Attach Policy
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecr-public:GetAuthorizationToken",
"ecr-public:InitiateLayerUpload",
"ecr-public:UploadLayerPart",
"ecr-public:CompleteLayerUpload",
"ecr-public:PutImage"
],
"Resource": "*"
}
]
}
```
```bash
aws iam put-user-policy \
--user-name gitea-ci-ecr-public \
--policy-name ECRPublicPush \
--policy-document file://ecr-public-push-policy.json
```
### 1.5 Create Access Keys
```bash
aws iam create-access-key --user-name gitea-ci-ecr-public
# Save: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
```
---
## 2. Gitea Secrets Configuration
Go to: **Repository → Settings → Secrets → Actions**
| Secret Name | Value |
|-------------|-------|
| `AWS_ACCESS_KEY_ID` | From step 1.5 |
| `AWS_SECRET_ACCESS_KEY` | From step 1.5 |
| `ECR_PUBLIC_REGISTRY_ALIAS` | Your registry alias (e.g., `vkist-pilot`) |
| `WEBHOOK_SECRET` | `openssl rand -hex 32` |
| `DEPLOY_WEBHOOK_URL` | `http://<lightsail-ip>:3333/deploy` |
---
## 3. Lightsail VM Setup
### 3.1 Install Docker + Compose
```bash
sudo apt update
sudo apt install -y docker.io docker-compose-plugin python3
sudo usermod -aG docker ubuntu
newgrp docker
```
### 3.2 Deploy Project Files
```bash
cd /opt
sudo git clone <your-gitea-repo-url> pilot-project
sudo chown -R ubuntu:ubuntu pilot-project
cd pilot-project
```
### 3.3 Configure Environment
```bash
cp workspace/sprint_1_2/CODEBASE/deps/spec/env.example .env
# Edit .env with your values:
# ECR_PUBLIC_REGISTRY_ALIAS=vkist-pilot
# WEBHOOK_SECRET=<same-as-gitea-secret>
```
### 3.4 Deploy Compose File
```bash
cp workspace/sprint_1_2/CODEBASE/deps/spec/docker-compose.frontend.yaml .
# Create external network
docker network create pilot-network
```
### 3.5 Install Deploy Script + Webhook Listener
```bash
cp workspace/sprint_1_2/CODEBASE/deps/spec/webhook-deploy.sh .
cp workspace/sprint_1_2/CODEBASE/deps/spec/webhook-listener.py .
chmod +x webhook-deploy.sh
```
### 3.6 Install Systemd Service
```bash
sudo cp workspace/sprint_1_2/CODEBASE/deps/spec/gitea-webhook-listener.service \
/etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable gitea-webhook-listener
sudo systemctl start gitea-webhook-listener
sudo systemctl status gitea-webhook-listener
```
### 3.7 Verify Webhook Listener
```bash
curl -X POST http://localhost:3333/deploy \
-H "Content-Type: application/json" \
-H "X-Gitea-Signature: sha256=$(echo -n '{}' | openssl dgst -sha256 -hmac "$(grep WEBHOOK_SECRET .env | cut -d= -f2)" | cut -d' ' -f2)" \
-d '{}'
# Should return "Event ignored" (no workflow completed)
```
---
## 4. Gitea Webhook Configuration
### 4.1 Add Webhook
**Repository → Settings → Webhooks → Add Webhook**
| Field | Value |
|-------|-------|
| Target URL | `http://<lightsail-ip>:3333/deploy` |
| Secret | `<same WEBHOOK_SECRET from step 2>` |
| Events | ✅ Workflow Run |
| Active | ✅ |
### 4.2 Test Webhook
Use Gitea's "Test Delivery" button → should return 200 with "Event ignored"
---
## 5. Caddy Reverse Proxy (Auto-HTTPS)
### 5.1 Install Caddy
```bash
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy
```
### 5.2 Configure Caddyfile
```bash
# Copy Caddyfile template
cp workspace/sprint_1_2/CODEBASE/deps/spec/Caddyfile.example /opt/pilot-project/Caddyfile
# Edit with your domain
sudo vim /opt/pilot-project/Caddyfile
```
### 5.3 Caddyfile Content
```text
# /opt/pilot-project/Caddyfile
# Replace app.example.com with your actual domain
app.example.com {
reverse_proxy lumina-frontend:80
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "SAMEORIGIN"
Referrer-Policy "strict-origin-when-cross-origin"
}
@static {
path *.js *.css *.wasm *.png *.jpg *.svg *.woff2 *.ico *.map
}
header @static Cache-Control "public, max-age=31536000, immutable"
}
```
### 5.4 Run Caddy (Docker Compose - Recommended)
Add to your `docker-compose.frontend.yaml` or create separate `docker-compose.caddy.yaml`:
```yaml
# docker-compose.caddy.yaml
version: '3.8'
services:
caddy:
image: caddy:2-alpine
container_name: caddy
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- /opt/pilot-project/Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
- caddy_config:/config
networks:
- pilot-network
volumes:
caddy_data:
caddy_config:
networks:
pilot-network:
external: true
```
```bash
# Start Caddy
docker compose -f docker-compose.caddy.yaml up -d
# Or run directly on host (if not using Docker)
# sudo systemctl enable --now caddy
```
### 5.5 Verify HTTPS
```bash
# Wait ~30s for Let's Encrypt cert issuance
curl -I https://app.example.com/health
# Should return 200 with strict-transport-security header
```
---
## 6. First Deployment Test
### 6.1 Trigger CI
```bash
cd <local-repo>
# Make a small change
echo "# test" >> workspace/sprint_1_2/CODEBASE/frontend/implementation/README.md
git add . && git commit -m "test: trigger CI" && git push origin main
```
### 6.2 Monitor
- **Gitea Actions tab** → Watch workflow run
- **Lightsail** → `journalctl -u gitea-webhook-listener -f`
- **ECR Public Console** → Verify image pushed
### 6.3 Verify
```bash
# Via Caddy (HTTPS)
curl https://app.example.com/health
# Should return "healthy"
# Direct to frontend (HTTP, for debugging)
curl http://<lightsail-ip>:8080/health
# Should return "healthy"
```
---
## 7. Rollback Procedure
```bash
# On Lightsail
docker images public.ecr.aws/vkist-pilot/lumina-frontend
# Tag previous SHA as latest
docker tag public.ecr.aws/vkist-pilot/lumina-frontend:<old-sha> \
public.ecr.aws/vkist-pilot/lumina-frontend:latest
# Redeploy
cd /opt/pilot-project
docker compose -f docker-compose.frontend.yaml up -d --force-recreate lumina-frontend
```
---
## 8. Cost Estimate (Monthly)
| Component | Cost |
|-----------|------|
| Lightsail 2 GB | $10 |
| ECR Public storage (1 GB) | ~$0.10 |
| ECR Public data transfer | Free (public pulls) |
| **Total** | **~$10.10/month** |
---
## 9. Troubleshooting
| Issue | Check |
|-------|-------|
| CI fails at ECR login | AWS credentials in Gitea secrets, IAM policy |
| Image not found on pull | Registry alias correct, image pushed to ECR |
| Webhook 401 | `WEBHOOK_SECRET` matches in .env and Gitea |
| Container unhealthy | `docker logs lumina-frontend`, check nginx config |
| Caddy cert not issued | Port 80/443 open on Lightsail firewall, DNS points to Lightsail IP |
| OOM on 2 GB VM | Reduce `memory: 128m` in compose, add swap |
---
## 10. File Inventory
| File | Location | Purpose |
|------|----------|---------|
| `frontend.yaml` | `.gitea/workflows/` | CI workflow |
| `Dockerfile` | `workspace/sprint_1_2/CODEBASE/frontend/implementation/` | Multi-stage build |
| `nginx.conf` | `workspace/sprint_1_2/CODEBASE/frontend/implementation/` | SPA routing + caching (inside container) |
| `docker-compose.frontend.yaml` | `workspace/sprint_1_2/CODEBASE/deps/spec/` | Production runtime |
| `webhook-deploy.sh` | `workspace/sprint_1_2/CODEBASE/deps/spec/` | Deploy script |
| `webhook-listener.py` | `workspace/sprint_1_2/CODEBASE/deps/spec/` | Webhook HTTP server |
| `gitea-webhook-listener.service` | `workspace/sprint_1_2/CODEBASE/deps/spec/` | Systemd unit |
| `env.example` | `workspace/sprint_1_2/CODEBASE/deps/spec/` | Environment template |
| `Caddyfile.example` | `workspace/sprint_1_2/CODEBASE/deps/spec/` | Caddy config template |
| `ci_cd_docker_registry_flow.md` | `workspace/sprint_1_2/CODEBASE/deps/spec/` | Full documentation |
| `ci_cd_ecr_public_flow.md` | `workspace/sprint_1_2/CODEBASE/deps/spec/` | ECR Public specific docs |
---
*Generated for VKIST Pilot Project — MSK Ultrasound Stack*
*Compatible with: Gitea 1.21+, Docker 24+, Node 20, AWS ECR Public, Caddy 2*

478
workspace/sprint_1_2/CODEBASE/deps/spec/ci_cd_frontend/ci_cd_docker_registry_flow.md Normal file
View File

@@ -0,0 +1,478 @@
# CI/CD Flow: Docker Registry-Based Deployment (with Caddy)
## Overview
This document specifies the complete CI/CD flow where the Gitea CI pipeline builds the frontend Docker image, pushes it to a container registry, and the Lightsail VM pulls and deploys the image via Docker Compose. **Caddy** handles automatic HTTPS termination and reverse proxying.
---
## Architecture Diagram
```mermaid
sequenceDiagram
autonumber
actor Dev as Developer
participant Git as Local Git
participant Gitea as Gitea (Lightsail VM)
participant CI as Gitea Actions Runner
participant Registry as Container Registry
participant Lightsail as Lightsail VM (Production)
participant Caddy as Caddy (HTTPS + Reverse Proxy)
Dev->>Git: git push origin main
Git->>Gitea: Push commits
Note over Gitea: Receives push event
Gitea->>CI: Dispatch workflow job
par CI Build Stage
CI->>CI: Checkout code (actions/checkout@v4)
CI->>CI: Setup Node.js 26 (actions/setup-node@v4)
CI->>CI: npm ci (install deps)
CI->>CI: npm run build (verify build)
CI->>CI: Login to registry (docker/login-action)
CI->>CI: Build Docker image (docker/build-push-action)
CI->>Registry: Push image (tagged with SHA + latest)
end
Note over Registry: Image stored: registry.example.com/lumina-frontend:sha123
par Deploy Stage
Lightsail->>Registry: Pull latest image
Lightsail->>Lightsail: docker compose pull
Lightsail->>Lightsail: docker compose up -d --force-recreate
Lightsail->>Caddy: Automatic HTTPS (no reload needed)
end
Dev->>Caddy: Access https://app.example.com
Caddy->>Lightsail: Proxy to frontend container (localhost:8080)
Lightsail->>Dev: Serves frontend
```
---
## Component Diagram (C4)
```mermaid
C4Container
title Container Diagram — Docker Registry CI/CD with Caddy
Person(dev, "Developer", "Writes code, pushes to Gitea")
System_Boundary(gitea_boundary, "Gitea Server (Lightsail VM)") {
Container(gitea, "Gitea", "Git + CI/CD", "Hosts repos, runs Actions workflows")
Container(runner, "Gitea Actions Runner", "Ephemeral", "Executes CI jobs in containers")
}
System_Boundary(registry_boundary, "Container Registry") {
Container(registry, "Registry", "Docker Hub / GCR / ECR / Harbor", "Stores built images")
}
System_Boundary(prod_boundary, "Production (Lightsail VM)") {
Container(compose, "Docker Compose", "Orchestrator", "Manages frontend container lifecycle")
Container(frontend, "Frontend Container", "nginx:alpine", "Serves Lumina MSK static assets")
Container(caddy, "Caddy", "HTTPS + Reverse Proxy", "Auto-TLS via Let's Encrypt, proxies to frontend")
}
Rel(dev, gitea, "git push", "HTTPS/SSH")
Rel(gitea, runner, "Dispatches job", "Internal API")
Rel(runner, registry, "docker push", "HTTPS")
Rel(compose, registry, "docker pull", "HTTPS")
Rel(compose, frontend, "Manages container", "Docker API")
Rel(caddy, frontend, "Reverse proxy", "HTTP (localhost:8080)")
Rel(dev, caddy, "Accesses app", "HTTPS (auto-TLS)")
```
---
## Flow Description
### 1. Trigger Phase
| Event | Condition | Workflow File |
|-------|-----------|---------------|
| Push to `main` | Changes under `workspace/sprint_1_2/CODEBASE/frontend/implementation/**` | `.gitea/workflows/frontend.yaml` |
| Manual dispatch | Via Gitea UI | Same workflow |
### 2. CI Build Stage (Runs on Gitea Actions Runner)
| Step | Tool | Purpose |
|------|------|---------|
| Checkout | `actions/checkout@v4` | Clone repository |
| Setup Node | `actions/setup-node@v4` | Node.js 26 + cache `node_modules` |
| Install deps | `npm ci` | Clean, reproducible install |
| Verify build | `npm run build` | Runs `sync-wasm` → `tsc -b` → `vite build`; fails if TypeScript errors |
| Login registry | `docker/login-action@v3` | Authenticate to registry using secrets |
| Build & push | `docker/build-push-action@v5` | Multi-stage build, push with SHA + `latest` tags |
**Image tags pushed:**
- `registry.example.com/lumina-frontend:<git-sha>` — immutable, traceable
- `registry.example.com/lumina-frontend:latest` — rolling pointer for deploy
### 3. Registry
Supported registries (pick one):
- **Docker Hub** — public/private, generous free tier
- **GitHub Container Registry (GHCR)** — integrated with GitHub, but Gitea can push
- **Google Container Registry (GCR)** — if on GCP
- **Amazon ECR Public** — free public images, authenticated pushes
- **Harbor / self-hosted** — full control, on Lightsail or separate VM
**Required secrets in Gitea:**
| Secret Name | Value |
|-------------|-------|
| `REGISTRY_URL` | e.g., `docker.io` or `ghcr.io` |
| `REGISTRY_USER` | Registry username |
| `REGISTRY_PASS` | Registry password / token |
### 4. Deploy Stage (Runs on Lightsail VM)
Two sub-options:
#### 4a. Polling / Cron (Simplest)
```bash
# /etc/cron.hourly/deploy-frontend
#!/bin/bash
cd /opt/pilot-project
docker compose -f docker-compose.frontend.yaml pull lumina-frontend
docker compose -f docker-compose.frontend.yaml up -d --force-recreate lumina-frontend
docker image prune -f
```
#### 4b. Webhook Listener (Event-driven)
A small HTTP server on Lightsail receives a webhook from Gitea after CI succeeds and triggers deploy immediately.
```bash
# webhook-deploy.sh (triggered by webhook)
cd /opt/pilot-project
git pull # optional: ensure compose file is current
docker compose -f docker-compose.frontend.yaml pull lumina-frontend
docker compose -f docker-compose.frontend.yaml up -d --force-recreate lumina-frontend
```
Gitea webhook config:
- **URL**: `http://<lightsail-ip>:3333/deploy`
- **Events**: Workflow run completed (success)
- **Secret**: Shared HMAC secret
### 5. Runtime on Lightsail (with Caddy)
#### Docker Compose
```yaml
# docker-compose.frontend.yaml
version: '3.8'
services:
lumina-frontend:
image: registry.example.com/lumina-frontend:latest
container_name: lumina-frontend
restart: unless-stopped
expose:
- "80" # Internal only, Caddy proxies to this
deploy:
resources:
limits:
memory: 128m
cpus: '0.5'
healthcheck:
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost/health"]
interval: 30s
timeout: 5s
retries: 3
caddy:
image: caddy:2-alpine
container_name: caddy
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "443:443/udp" # HTTP/3
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
- caddy_config:/config
depends_on:
- lumina-frontend
volumes:
caddy_data:
caddy_config:
networks:
default:
name: pilot-network
external: true
```
#### Caddyfile (Auto-HTTPS)
```text
# Caddyfile
# Replace with your actual domain
app.example.com {
# Reverse proxy to frontend container
reverse_proxy lumina-frontend:80
# Optional: Security headers
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "SAMEORIGIN"
Referrer-Policy "strict-origin-when-cross-origin"
}
# Cache static assets
@static {
path *.js *.css *.wasm *.png *.jpg *.svg *.woff2 *.ico *.map
}
header @static Cache-Control "public, max-age=31536000, immutable"
}
```
**Key benefits of Caddy:**
- **Automatic HTTPS** — No manual cert management, Let's Encrypt handled automatically
- **HTTP/2 & HTTP/3** — Enabled by default
- **Zero-downtime reloads** — Config changes don't drop connections
- **No external ACME client needed** — Built-in
---
## Required Artifacts to Create
| File | Location | Purpose |
|------|----------|---------|
| `frontend.yaml` | `.gitea/workflows/` | CI workflow definition |
| `Dockerfile` | `workspace/sprint_1_2/CODEBASE/frontend/implementation/` | Multi-stage build |
| `docker-compose.frontend.yaml` | `/opt/pilot-project/` on Lightsail | Production runtime (frontend + Caddy) |
| `Caddyfile` | `/opt/pilot-project/` on Lightsail | Caddy config with domain |
| `webhook-deploy.sh` (optional) | `/opt/pilot-project/` on Lightsail | Deploy script |
| systemd unit (optional) | `/etc/systemd/system/` on Lightsail | Run webhook listener |
| Gitea secrets | Gitea UI → Repo → Settings → Secrets | Registry credentials |
---
## Secrets Management
| Secret | Where | Scope |
|--------|-------|-------|
| `REGISTRY_URL` | Gitea repo secrets | CI build step |
| `REGISTRY_USER` | Gitea repo secrets | CI build step |
| `REGISTRY_PASS` | Gitea repo secrets | CI build step |
| `WEBHOOK_SECRET` | Gitea webhook config + Lightsail env | Deploy trigger auth |
---
## Failure Scenarios & Mitigations
| Scenario | Detection | Mitigation |
|----------|-----------|------------|
| CI build fails | Workflow red in Gitea | Fix code, re-push |
| Registry push fails | CI step fails | Check credentials, quota |
| Pull fails on Lightsail | Deploy script logs | Check network, image tag exists |
| Container crash loop | Healthcheck fails | `docker logs lumina-frontend` |
| Caddy cert failure | Caddy logs (`docker logs caddy`) | Check domain DNS, port 80/443 reachable |
| OOM on 2 GB VM | `free -h`, `docker stats` | Reduce `memory` limit, add swap |
---
## Rollback
```bash
# On Lightsail: list available tags
docker images registry.example.com/lumina-frontend
# Rollback to specific SHA
docker tag registry.example.com/lumina-frontend:<old-sha> registry.example.com/lumina-frontend:latest
docker compose -f docker-compose.frontend.yaml up -d --force-recreate lumina-frontend
```
---
## Cost Estimate (Monthly)
| Component | Estimate |
|-----------|----------|
| Lightsail 2 GB instance | $10–12 |
| Registry (Docker Hub free / GHCR) | $0–5 |
| Data transfer | < $1 |
| **Total** | **~$10–18 / month** |
---
## Next Steps
1. Choose a container registry and create repo/namespace
2. Add registry secrets to Gitea
3. Create `.gitea/workflows/frontend.yaml` (see appendix)
4. Create `Dockerfile` in frontend implementation dir
5. Deploy `docker-compose.frontend.yaml` + `Caddyfile` to Lightsail
6. Point domain DNS to Lightsail public IP
7. Test end-to-end: push to `main` → verify CI → verify deploy → verify HTTPS
---
## Appendix: Minimal Workflow File
```yaml
# .gitea/workflows/frontend.yaml
name: Frontend CI/CD
on:
push:
branches: [main]
paths:
- 'workspace/sprint_1_2/CODEBASE/frontend/implementation/**'
workflow_dispatch: {}
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: workspace/sprint_1_2/CODEBASE/frontend/implementation/package-lock.json
- name: Install dependencies
run: npm ci
working-directory: ./workspace/sprint_1_2/CODEBASE/frontend/implementation
- name: Build frontend (verify)
run: npm run build
working-directory: ./workspace/sprint_1_2/CODEBASE/frontend/implementation
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ secrets.REGISTRY_URL }}
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASS }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ secrets.REGISTRY_URL }}/${{ secrets.REGISTRY_USER }}/lumina-frontend
tags: |
type=ref,event=branch
type=sha,prefix=
type=raw,value=latest,enable={{is_default_branch}}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: ./workspace/sprint_1_2/CODEBASE/frontend/implementation
file: ./workspace/sprint_1_2/CODEBASE/frontend/implementation/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
```
---
## Appendix: Minimal Dockerfile
```dockerfile
# workspace/sprint_1_2/CODEBASE/frontend/implementation/Dockerfile
# Stage 1: Build
FROM node:20-bookworm-slim AS builder
WORKDIR /app
COPY package.json package-lock.json* ./
COPY scripts/ ./scripts/
RUN npm ci
COPY . .
RUN npm run build
# Stage 2: Runtime
FROM nginx:alpine
RUN rm -rf /usr/share/nginx/html/*
COPY --from=builder /app/dist /usr/share/nginx/html
# Custom nginx config for SPA + caching
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
```
---
## Appendix: nginx.conf (for Frontend Container)
```nginx
# workspace/sprint_1_2/CODEBASE/frontend/implementation/nginx.conf
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Cache static assets aggressively
location ~* \.(js|css|wasm|png|jpg|jpeg|svg|woff2|ico|map)$ {
expires 1y;
add_header Cache-Control "public, immutable";
access_log off;
}
# SPA fallback
location / {
try_files $uri $uri/ /index.html;
}
# Health check endpoint
location /health {
access_log off;
return 200 "healthy\n";
add_header Content-Type text/plain;
}
# Gzip compression
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_proxied any;
gzip_types text/plain text/css text/xml text/javascript application/javascript application/xml application/json application/wasm;
}
```
---
## Appendix: Caddyfile (for Lightsail)
```text
# /opt/pilot-project/Caddyfile
# Replace app.example.com with your actual domain
app.example.com {
reverse_proxy lumina-frontend:80
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "SAMEORIGIN"
Referrer-Policy "strict-origin-when-cross-origin"
}
@static {
path *.js *.css *.wasm *.png *.jpg *.svg *.woff2 *.ico *.map
}
header @static Cache-Control "public, max-age=31536000, immutable"
}
```
---
*Document version: 2.0 (Caddy integration)*
*Last updated: 2026-07-17*
*Owner: Platform Engineering / VKIST Team*

756
workspace/sprint_1_2/CODEBASE/deps/spec/ci_cd_frontend/ci_cd_ecr_public_flow.md Normal file
View File

@@ -0,0 +1,756 @@
# CI/CD Flow: Docker Registry-Based Deployment (Amazon ECR Public)
## Overview
This document specifies the complete CI/CD flow where the Gitea CI pipeline builds the frontend Docker image, pushes it to **Amazon ECR Public**, and the Lightsail VM pulls and deploys the image via Docker Compose with **Caddy** as the reverse proxy (auto-HTTPS via Let's Encrypt).
---
## Architecture Diagram
```mermaid
sequenceDiagram
autonumber
actor Dev as Developer
participant Git as Local Git
participant Gitea as Gitea (Lightsail VM)
participant CI as Gitea Actions Runner
participant ECR as Amazon ECR Public
participant Lightsail as Lightsail VM (Production)
participant Caddy as Caddy (Reverse Proxy + TLS)
Dev->>Git: git push origin main
Git->>Gitea: Push commits
Note over Gitea: Receives push event
Gitea->>CI: Dispatch workflow job
par CI Build Stage
CI->>CI: Checkout code (actions/checkout@v4)
CI->>CI: Configure AWS credentials (aws-actions/configure-aws-credentials)
CI->>CI: Login to ECR Public (aws-actions/amazon-ecr-login)
CI->>CI: Setup Node.js 20 (actions/setup-node@v4)
CI->>CI: npm ci (install deps)
CI->>CI: npm run build (verify build)
CI->>CI: Build Docker image (docker/build-push-action)
CI->>ECR: Push image (tagged with SHA + latest)
end
Note over ECR: Image stored: public.ecr.aws/<alias>/lumina-frontend:sha123
par Deploy Stage
Lightsail->>ECR: Pull latest image (public, no auth needed for pull)
Lightsail->>Lightsail: docker compose pull
Lightsail->>Lightsail: docker compose up -d --force-recreate
Lightsail->>Caddy: Reload config (if needed)
end
Dev->>Lightsail: Access https://app.example.com
Lightsail->>Dev: Serves frontend (auto-HTTPS)
```
---
## Component Diagram (C4)
```mermaid
C4Container
title Container Diagram — ECR Public CI/CD with Caddy
Person(dev, "Developer", "Writes code, pushes to Gitea")
System_Boundary(gitea_boundary, "Gitea Server (Lightsail VM)") {
Container(gitea, "Gitea", "Git + CI/CD", "Hosts repos, runs Actions workflows")
Container(runner, "Gitea Actions Runner", "Ephemeral", "Executes CI jobs in containers")
}
System_Boundary(aws_boundary, "AWS Cloud") {
Container(ecr, "Amazon ECR Public", "public.ecr.aws", "Stores public container images")
}
System_Boundary(prod_boundary, "Production (Lightsail VM)") {
Container(compose, "Docker Compose", "Orchestrator", "Manages frontend container lifecycle")
Container(caddy, "Caddy", "Reverse Proxy + TLS", "Auto-HTTPS via Let's Encrypt, serves Lumina MSK frontend")
}
Rel(dev, gitea, "git push", "HTTPS/SSH")
Rel(gitea, runner, "Dispatches job", "Internal API")
Rel(runner, ecr, "docker push (authenticated)", "HTTPS + AWS SigV4")
Rel(compose, ecr, "docker pull (public)", "HTTPS")
Rel(compose, caddy, "Proxies to frontend", "Docker network")
Rel(dev, caddy, "Accesses app", "HTTPS (auto)")
```
---
## Flow Description
### 1. Trigger Phase
| Event | Condition | Workflow File |
|-------|-----------|---------------|
| Push to `main` | Changes under `workspace/sprint_1_2/CODEBASE/frontend/implementation/**` | `.gitea/workflows/frontend.yaml` |
| Manual dispatch | Via Gitea UI | Same workflow |
### 2. CI Build Stage (Runs on Gitea Actions Runner)
| Step | Tool | Purpose |
|------|------|---------|
| Checkout | `actions/checkout@v4` | Clone repository |
| Configure AWS | `aws-actions/configure-aws-credentials@v4` | Set AWS credentials for ECR access |
| Login to ECR Public | `aws-actions/amazon-ecr-login@v2` | Authenticate Docker to ECR Public |
| Setup Node | `actions/setup-node@v4` | Node.js 20 + cache `node_modules` |
| Install deps | `npm ci` | Clean, reproducible install |
| Verify build | `npm run build` | Runs `sync-wasm` → `tsc -b` → `vite build`; fails if TypeScript errors |
| Build & push | `docker/build-push-action@v5` | Multi-stage build, push with SHA + `latest` tags |
**Image tags pushed:**
- `public.ecr.aws/<registry-alias>/lumina-frontend:<git-sha>` — immutable, traceable
- `public.ecr.aws/<registry-alias>/lumina-frontend:latest` — rolling pointer for deploy
### 3. Amazon ECR Public
**Repository URI format:**
```
public.ecr.aws/<registry-alias>/lumina-frontend
```
**Key properties:**
- **Public pulls** — No authentication required for `docker pull`
- **Authenticated pushes** — Requires AWS credentials with `ecr-public:PutImage`, `ecr-public:InitiateLayerUpload`, `ecr-public:UploadLayerPart`, `ecr-public:CompleteLayerUpload`
- **Registry alias** — Created once per AWS account (e.g., `vkist-pilot`)
- **Region** — `us-east-1` (ECR Public is only in us-east-1)
**Required IAM permissions for CI user/role:**
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecr-public:GetAuthorizationToken",
"ecr-public:InitiateLayerUpload",
"ecr-public:UploadLayerPart",
"ecr-public:CompleteLayerUpload",
"ecr-public:PutImage"
],
"Resource": "*"
}
]
}
```
**Required secrets in Gitea:**
| Secret Name | Value |
|-------------|-------|
| `AWS_ACCESS_KEY_ID` | IAM user access key with ECR Public permissions |
| `AWS_SECRET_ACCESS_KEY` | Corresponding secret key |
| `AWS_ECR_REGION` | `us-east-1` (ECR Public region) |
| `ECR_PUBLIC_REGISTRY_ALIAS` | Your ECR Public registry alias (e.g., `vkist-pilot`) |
### 4. Deploy Stage (Runs on Lightsail VM)
Since ECR Public images are publicly pullable, **no AWS credentials needed on Lightsail** for deployment.
Two sub-options:
#### 4a. Polling / Cron (Simplest)
```bash
#!/bin/bash
# /etc/cron.hourly/deploy-frontend
cd /opt/pilot-project
docker compose -f docker-compose.frontend.yaml pull lumina-frontend
docker compose -f docker-compose.frontend.yaml up -d --force-recreate lumina-frontend
docker image prune -f
```
#### 4b. Webhook Listener (Event-driven)
A small HTTP server on Lightsail receives a webhook from Gitea after CI succeeds and triggers deploy immediately.
```bash
#!/bin/bash
# /opt/pilot-project/webhook-deploy.sh
set -e
cd /opt/pilot-project
git pull # optional: ensure compose file is current
docker compose -f docker-compose.frontend.yaml pull lumina-frontend
docker compose -f docker-compose.frontend.yaml up -d --force-recreate lumina-frontend
```
```python
#!/usr/bin/env python3
"""
Gitea Webhook Listener - secrets from AWS SSM Parameter Store
"""
import os
import hmac
import hashlib
import json
import subprocess
import logging
from pathlib import Path
from http.server import HTTPServer, BaseHTTPRequestHandler
try:
import boto3
except ImportError:
print("ERROR: boto3 required. Install: pip install boto3")
exit(1)
# Configuration
SSM_PREFIX = "/lumina"
AWS_ECR_REGION = os.environ.get("AWS_ECR_REGION", "us-east-1")
LISTEN_HOST = "127.0.0.1"
LISTEN_PORT = 3333
DEPLOY_SCRIPT = Path(__file__).parent / "webhook-deploy.sh"
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
logger = logging.getLogger(__name__)
def load_secrets_from_ssm() -> dict:
"""Fetch all parameters under /lumina prefix from SSM Parameter Store."""
client = boto3.client("ssm", region_name=AWS_ECR_REGION)
response = client.get_parameters_by_path(
Path=SSM_PREFIX,
WithDecryption=True,
Recursive=True
)
secrets = {}
for param in response.get("Parameters", []):
key = param["Name"].split("/")[-1].upper()
secrets[key] = param["Value"]
return secrets
SECRETS = load_secrets_from_ssm()
WEBHOOK_SECRET = SECRETS.get("WEBHOOK_SECRET", "")
ECR_REGISTRY_ALIAS = SECRETS.get("ECR_REGISTRY_ALIAS", "")
if not WEBHOOK_SECRET:
logger.error("WEBHOOK_SECRET not found in SSM at %s/webhook-secret", SSM_PREFIX)
exit(1)
def verify_signature(payload: bytes, signature_header: str) -> bool:
if not signature_header or not signature_header.startswith("sha256="):
return False
expected = "sha256=" + hmac.new(
WEBHOOK_SECRET.encode(), payload, hashlib.sha256
).hexdigest()
return hmac.compare_digest(expected, signature_header)
class WebhookHandler(BaseHTTPRequestHandler):
def do_POST(self):
if self.path != "/deploy":
self.send_response(404)
self.end_headers()
return
content_length = int(self.headers.get("Content-Length", 0))
payload = self.rfile.read(content_length)
signature = self.headers.get("X-Gitea-Signature", "")
if not verify_signature(payload, signature):
logger.warning("Invalid signature from %s", self.client_address[0])
self.send_response(401)
self.end_headers()
self.wfile.write(b"Invalid signature")
return
try:
event = json.loads(payload)
except json.JSONDecodeError:
self.send_response(400)
self.end_headers()
return
# Only deploy on successful frontend workflow
if event.get("workflow_run", {}).get("conclusion") != "success":
logger.info("Workflow not successful, ignoring")
self.send_response(200)
self.end_headers()
self.wfile.write(b"Ignored")
return
logger.info("Triggering deployment")
try:
result = subprocess.run(
[str(DEPLOY_SCRIPT)],
capture_output=True,
text=True,
timeout=300,
env={**os.environ, "ECR_PUBLIC_REGISTRY_ALIAS": ECR_REGISTRY_ALIAS}
)
if result.returncode == 0:
self.send_response(200)
self.end_headers()
self.wfile.write(b"Deployed")
else:
self.send_response(500)
self.end_headers()
self.wfile.write(f"Failed: {result.stderr}".encode())
except Exception as e:
self.send_response(500)
self.end_headers()
self.wfile.write(f"Error: {e}".encode())
def log_message(self, format, *args):
logger.info("%s - %s", self.client_address[0], format % args)
def main():
if not DEPLOY_SCRIPT.exists():
logger.error("Deploy script not found: %s", DEPLOY_SCRIPT)
exit(1)
os.chmod(DEPLOY_SCRIPT, 0o755)
server = HTTPServer((LISTEN_HOST, LISTEN_PORT), WebhookHandler)
logger.info("Listener on %s:%d (registry: %s)", LISTEN_HOST, LISTEN_PORT, ECR_REGISTRY_ALIAS)
try:
server.serve_forever()
except KeyboardInterrupt:
server.shutdown()
if __name__ == "__main__":
main()
```
**systemd unit** (`/etc/systemd/system/gitea-webhook-listener.service`):
```ini
[Unit]
Description=Gitea Webhook Listener (Lumina Frontend)
After=network.target docker.service
Requires=docker.service
[Service]
Type=simple
User=ubuntu
WorkingDirectory=/opt/pilot-project
ExecStart=/usr/bin/python3 /opt/pilot-project/webhook-listener.py
Restart=always
RestartSec=10
# Security
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/opt/pilot-project
ProtectHome=true
[Install]
WantedBy=multi-user.target
```
**Gitea webhook config:**
- **URL**: `http://<lightsail-public-ip>:3333/deploy` (or via Caddy proxy)
- **Events**: Workflow Run → Completed
- **Secret**: Same `WEBHOOK_SECRET` value (stored in SSM)
- **Content-Type**: `application/json`
### 5. Runtime on Lightsail with Caddy
**Docker Compose** (`docker-compose.frontend.yaml`):
```yaml
version: '3.8'
services:
lumina-frontend:
image: public.ecr.aws/${ECR_PUBLIC_REGISTRY_ALIAS}/lumina-frontend:latest
container_name: lumina-frontend
restart: unless-stopped
expose:
- "80"
deploy:
resources:
limits:
memory: 128m
cpus: '0.5'
healthcheck:
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost/health"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
caddy:
image: caddy:2-alpine
container_name: caddy
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "443:443/udp"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
- caddy_config:/config
depends_on:
- lumina-frontend
networks:
default:
name: pilot-network
external: true
volumes:
caddy_data:
caddy_config:
```
**Caddyfile** (`/opt/pilot-project/Caddyfile`):
```text
# Replace app.example.com with your actual domain
app.example.com {
reverse_proxy lumina-frontend:80
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "SAMEORIGIN"
Referrer-Policy "strict-origin-when-cross-origin"
}
@static {
path *.js *.css *.wasm *.png *.jpg *.svg *.woff2 *.ico *.map
}
header @static Cache-Control "public, max-age=31536000, immutable"
}
```
**Environment file** (`/opt/pilot-project/.env`):
```bash
ECR_PUBLIC_REGISTRY_ALIAS=vkist-pilot
```
---
## Required Artifacts to Create
| File | Location | Purpose |
|------|----------|---------|
| `frontend.yaml` | `.gitea/workflows/` | CI workflow definition |
| `Dockerfile` | `workspace/sprint_1_2/CODEBASE/frontend/implementation/` | Multi-stage build |
| `docker-compose.frontend.yaml` | `/opt/pilot-project/` on Lightsail | Production runtime (frontend + Caddy) |
| `Caddyfile` | `/opt/pilot-project/` on Lightsail | Caddy reverse proxy config |
| `webhook-deploy.sh` (optional) | `/opt/pilot-project/` on Lightsail | Deploy script |
| `webhook-listener.py` (optional) | `/opt/pilot-project/` on Lightsail | Webhook server (secrets from SSM) |
| systemd unit (optional) | `/etc/systemd/system/` on Lightsail | Run webhook listener |
| `.env` | `/opt/pilot-project/` on Lightsail | Registry alias variable |
| Gitea secrets | Gitea UI → Repo → Settings → Secrets | AWS credentials + alias |
| SSM Parameters | AWS Console → Systems Manager → Parameter Store | `WEBHOOK_SECRET`, `ECR_REGISTRY_ALIAS` |
---
## Secrets Management
| Secret | Where | Scope |
|--------|-------|-------|
| `AWS_ACCESS_KEY_ID` | Gitea repo secrets | CI: ECR authentication |
| `AWS_SECRET_ACCESS_KEY` | Gitea repo secrets | CI: ECR authentication |
| `AWS_ECR_REGION` | Gitea repo secrets | CI: `us-east-1` |
| `ECR_PUBLIC_REGISTRY_ALIAS` | Gitea repo secrets + Lightsail `.env` | CI tag + Compose image ref |
| `WEBHOOK_SECRET` | **AWS SSM SecureString** (`/lumina/webhook-secret`) | Deploy trigger auth |
| `ECR_REGISTRY_ALIAS` | **AWS SSM String** (`/lumina/ecr-registry-alias`) | Deploy script env var |
**Lightsail IAM Instance Profile** must have:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["ssm:GetParameter", "ssm:GetParametersByPath"],
"Resource": "arn:aws:ssm:us-east-1:<account-id>:parameter/lumina/*"
}
]
}
```
---
## ECR Public Setup Steps
### 1. Create ECR Public Repository
```bash
aws ecr-public create-repository \
--repository-name lumina-frontend \
--region us-east-1
```
Note the `registryAlias` from output (e.g., `vkist-pilot`).
### 2. Create IAM User for CI
```bash
aws iam create-user --user-name gitea-ci-ecr-public
aws iam attach-user-policy \
--user-name gitea-ci-ecr-public \
--policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPublicFullAccess
# Or use custom least-privilege policy above
aws iam create-access-key --user-name gitea-ci-ecr-public
```
Save the Access Key ID and Secret Access Key.
### 3. Add Secrets to Gitea
Gitea UI → Repository → Settings → Secrets → Add:
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `AWS_ECR_REGION` = `us-east-1`
- `ECR_PUBLIC_REGISTRY_ALIAS` = your alias (e.g., `vkist-pilot`)
### 4. Store Secrets in SSM Parameter Store
```bash
# Webhook secret (generate once)
aws ssm put-parameter \
--name "/lumina/webhook-secret" \
--value "$(openssl rand -hex 32)" \
--type "SecureString" \
--region us-east-1
# Registry alias (not sensitive, but convenient)
aws ssm put-parameter \
--name "/lumina/ecr-registry-alias" \
--value "vkist-pilot" \
--type "String" \
--region us-east-1
```
### 5. Attach IAM Instance Profile to Lightsail
AWS Console → Lightsail → Your Instance → Networking → Attach IAM instance profile → Select profile with SSM read access.
---
## Failure Scenarios & Mitigations
| Scenario | Detection | Mitigation |
|----------|-----------|------------|
| CI build fails | Workflow red in Gitea | Fix code, re-push |
| ECR login fails | CI step fails | Check AWS credentials, permissions |
| Push fails (throttling) | CI step fails | Retry, check ECR Public rate limits |
| Pull fails on Lightsail | Deploy script logs | Check network, image tag exists |
| Container crash loop | Healthcheck fails | `docker logs lumina-frontend` |
| OOM on 2 GB VM | `free -h`, `docker stats` | Reduce `memory` limit, add swap |
| Caddy TLS fails | Caddy logs | Check DNS, port 80/443 reachable |
---
## Rollback
```bash
# On Lightsail: list available tags (requires AWS CLI for ECR Public)
aws ecr-public describe-images \
--repository-name lumina-frontend \
--region us-east-1 \
--query 'imageDetails[*].imageTags' \
--output text
# Rollback to specific SHA
docker tag public.ecr.aws/vkist-pilot/lumina-frontend:<old-sha> public.ecr.aws/vkist-pilot/lumina-frontend:latest
docker compose -f docker-compose.frontend.yaml up -d --force-recreate lumina-frontend
```
---
## Cost Estimate (Monthly)
| Component | Estimate |
|-----------|----------|
| Lightsail 2 GB instance | $10–12 |
| ECR Public storage (first 50 GB) | Free |
| ECR Public data transfer (first 5 TB/mo to internet) | Free |
| Data transfer (Lightsail ↔ ECR Public) | < $1 |
| **Total** | **~$10–13 / month** |
---
## Next Steps
1. Create ECR Public repository `lumina-frontend` in `us-east-1`
2. Create IAM user `gitea-ci-ecr-public` with ECR Public permissions
3. Add AWS credentials + registry alias to Gitea secrets
4. Store `WEBHOOK_SECRET` and `ECR_REGISTRY_ALIAS` in AWS SSM Parameter Store
5. Create `.gitea/workflows/frontend.yaml` (see appendix)
6. Create `Dockerfile` in frontend implementation dir
7. Deploy `docker-compose.frontend.yaml`, `Caddyfile`, `.env` to Lightsail
8. Attach IAM instance profile with SSM read access to Lightsail
9. Test end-to-end: push to `main` → verify CI → verify deploy + HTTPS
---
## Appendix: Workflow File for ECR Public
```yaml
# .gitea/workflows/frontend.yaml
name: Frontend CI/CD (ECR Public)
on:
push:
branches: [main]
paths:
- 'workspace/sprint_1_2/CODEBASE/frontend/implementation/**'
workflow_dispatch: {}
permissions:
contents: read
id-token: write
jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_ECR_REGION }}
- name: Login to Amazon ECR Public
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2
with:
registry-type: public
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: workspace/sprint_1_2/CODEBASE/frontend/implementation/package-lock.json
- name: Install dependencies
run: npm ci
working-directory: ./workspace/sprint_1_2/CODEBASE/frontend/implementation
- name: Build frontend (verify)
run: npm run build
working-directory: ./workspace/sprint_1_2/CODEBASE/frontend/implementation
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: public.ecr.aws/${{ secrets.ECR_PUBLIC_REGISTRY_ALIAS }}/lumina-frontend
tags: |
type=ref,event=branch
type=sha,prefix=
type=raw,value=latest,enable={{is_default_branch}}
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: ./workspace/sprint_1_2/CODEBASE/frontend/implementation
file: ./workspace/sprint_1_2/CODEBASE/frontend/implementation/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
```
---
## Appendix: Minimal Dockerfile
```dockerfile
# workspace/sprint_1_2/CODEBASE/frontend/implementation/Dockerfile
# Stage 1: Build
FROM node:20-bookworm-slim AS builder
WORKDIR /app
COPY package.json package-lock.json* ./
COPY scripts/ ./scripts/
RUN npm ci
COPY . .
RUN npm run build
# Stage 2: Runtime
FROM nginx:alpine
RUN rm -rf /usr/share/nginx/html/*
COPY --from=builder /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
```
---
## Appendix: nginx.conf for SPA (inside container)
```nginx
# workspace/sprint_1_2/CODEBASE/frontend/implementation/nginx.conf
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Cache static assets aggressively
location ~* \.(js|css|wasm|png|jpg|jpeg|svg|woff2|ico|map)$ {
expires 1y;
add_header Cache-Control "public, immutable";
access_log off;
}
# Main location - SPA fallback
location / {
try_files $uri $uri/ /index.html;
}
# Health check endpoint
location /health {
access_log off;
return 200 "healthy\n";
add_header Content-Type text/plain;
}
# Gzip compression
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_proxied any;
gzip_types text/plain text/css text/xml text/javascript application/javascript application/xml application/json application/wasm;
}
```
---
## Appendix: Caddyfile (Lightsail)
```text
# /opt/pilot-project/Caddyfile
# Replace app.example.com with your actual domain
app.example.com {
reverse_proxy lumina-frontend:80
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "SAMEORIGIN"
Referrer-Policy "strict-origin-when-cross-origin"
}
@static {
path *.js *.css *.wasm *.png *.jpg *.svg *.woff2 *.ico *.map
}
header @static Cache-Control "public, max-age=31536000, immutable"
}
```
---
*Document version: 2.0 (Caddy + SSM integration)*
*Last updated: 2026-07-17*
*Registry: Amazon ECR Public (us-east-1)*
*Reverse Proxy: Caddy (auto-HTTPS via Let's Encrypt)*
*Secrets: AWS SSM Parameter Store*
*Owner: Platform Engineering / VKIST Team*

0
workspace/sprint_1_2/CODEBASE/deps/spec/build_server_plan.md → workspace/sprint_1_2/CODEBASE/deps/spec/runner_build/build_server_plan.md
View File

0
workspace/sprint_1_2/CODEBASE/deps/spec/secrets_management_guideline.md → workspace/sprint_1_2/CODEBASE/deps/spec/secret_manage/secrets_management_guideline.md
View File

0
workspace/sprint_1_2/CODEBASE/deps/implementation/test_worker.py → workspace/sprint_1_2/CODEBASE/deps/test/test_worker.py
View File