include the test gitea secrets

workspace/sprint_1_2/CODEBASE/deps/spec/build_server_plan.md

@@ -0,0 +1,142 @@
+
+* The build was for creating the build pipeline in the CI/CD system of the project
+that involve: Lightsail instance as broker, Modal-Serverless as the build-worker, and Gitea as the source repository hosted on the same Lightsail VM.
+
+```mermaid
+sequenceDiagram
+    autonumber
+    actor Developer
+    participant Git as Codebase (Local Git)
+    participant Gitea as Gitea (on Lightsail VM)
+    participant Webhook as Webhook Listener (Lightsail VM)
+    participant Modal as Modal Serverless (VM Sandbox)
+
+    Developer->>Git: git push code
+    Git->>Gitea: Push commits to remote repo (Lightsail)
+    Note over Gitea: Gitea registers a new pending job<br/>in the Actions queue
+    Gitea->>Webhook: HTTP POST Webhook (Push event)
+    
+    rect rgb(240, 248, 255)
+        Note over Webhook: Triggered by Webhook (same Lightsail VM)
+        Webhook->>Modal: Spawn sandbox container via Modal SDK
+    end
+
+    rect rgb(245, 245, 245)
+        Note over Modal: Boot Sandbox VM (<2 seconds)
+        Modal->>Gitea: ./act_runner register (using registration token)
+        Modal->>Gitea: ./act_runner daemon --once (asks for work)
+        Gitea->>Modal: Delivers the queued build job
+        
+        Note over Modal: Runner clones codebase,<br/>builds Docker container, & runs test/compile
+        
+        Modal->>Gitea: Reports build results (Success/Failure)
+    end
+
+    Note over Modal: act_runner exits automatically (--once)<br/>Modal Sandbox self-destructs
+    Gitea->>Developer: Displays build status in UI
+```
+---
+
+### Phase 1: Gitea Preparation (Lightsail VM)
+
+This phase configures the Gitea instance running on the Lightsail VM to accept serverless runners.
+
+- [ ]  **Task 1.1: Enable Gitea Actions**
+    - SSH into the Lightsail VM.
+    - Open `/etc/gitea/app.ini` on the Lightsail instance.
+    - Add or update the following block to enable the Actions feature:Ini, TOML
+        
+        ```
+        [actions]
+        ENABLED = true
+        ```
+        
+    - Restart Gitea (`sudo systemctl restart gitea`).
+- [ ]  **Task 1.2: Retrieve Gitea Runner Registration Token**
+    - Access the Gitea web UI running on the Lightsail VM (e.g., `http://<lightsail-ip>:3000`) as an administrator.
+    - Navigate to **Site Administration** > **Actions** > **Runners**.
+    - Click **Registration Token** and copy this token (you will need it for the Modal Sandbox registration).
+
+> **Note:** All Gitea administration tasks in this plan assume you are managing the Gitea instance hosted on the Lightsail VM, either via SSH or its web UI.
+
+### Phase 2: Create the Webhook Listener (Lightsail VM)
+
+The listener runs on the same Lightsail VM as Gitea and catches the git push notification from Gitea to call Modal.
+
+- [ ]  **Task 2.1: Write the Express/Node.js Webhook Server**
+    - Create a simple script on the Lightsail VM that listens on a port (e.g., `3333`) and exposes a `/deploy` route.
+    - The route must verify Gitea's Webhook Secret header to ensure requests are authentic.
+    - When a valid POST request arrives, use the `child_process` module to run a local command: `modal run deploy_runner.py`.
+- [ ]  **Task 2.2: Configure Webhook in Gitea**
+    - In your Gitea repository settings (on the Lightsail-hosted Gitea), create a Gitea Webhook pointing to `http://127.0.0.1:3333/deploy` (or your Lightsail public domain).
+    - Set the trigger event to **Push Events** and assign a strong webhook secret token.
+
+### Phase 3: Build the Modal Serverless Runner (Modal Cloud)
+
+This python script defines the sandbox container and coordinates the registration, execution, and termination with the Gitea instance on Lightsail.
+
+- [ ]  **Task 3.1: Define the Modal Image (`deploy_runner.py`)**
+    - Configure a Modal image containing:
+        - `git`
+        - `docker` (or `podman`/`kaniko` if you are building Docker images inside the sandbox)
+        - The Gitea `act_runner` binary (downloaded directly from Gitea's release page).
+        - **Build dependencies for the project:**
+            - Node.js 18+ and npm (for frontend build)
+            - Python 3.10+ and pip (for any Python-based build steps)
+            - Build essentials (`build-essential`, `python3-dev`, `pkg-config`) for native module compilation
+            - `ca-certificates` and `curl` (for downloading dependencies and act_runner)
+- [ ]  **Task 3.2: Implement the Modal Function Logic**
+    - Set up a Modal function (e.g., `@app.function()`) that executes the following shell sequence inside the container when invoked:
+        1. **Register the runner:**Bash
+            
+            ```
+            ./act_runner register --no-interactive --instance <YOUR_GITEA_URL> --token <YOUR_REGISTRATION_TOKEN> --name modal-sandbox-runner --labels "ubuntu-latest:docker://node:18-bookworm"
+            ```
+            
+        2. **Run once and block until job completion:**Bash
+            
+            ```
+            ./act_runner daemon --once
+            ```
+            
+    > **Note:** `<YOUR_GITEA_URL>` must be the publicly reachable address of the Gitea instance on your Lightsail VM (e.g., `http://<lightsail-ip>:3000`), since the Modal sandbox runs outside of your Lightsail network.
+- [ ]  **Task 3.3: Deploy and Test the Modal App**
+    - Authenticate your Lightsail server with Modal using `modal setup` (run this on the Lightsail VM).
+    - Run `modal run deploy_runner.py` manually to verify the sandbox boots, registers in Gitea, checks for a job, and exits gracefully.
+
+### Phase 4: Configure the Repository CI/CD Workflow
+
+Define what the runner should actually do when it receives the codebase from the Lightsail-hosted Gitea.
+
+- [ ]  **Task 4.1: Create Gitea Action Workflow File**
+    - In your repository (hosted on the Lightsail Gitea instance), create a directory structure: `.gitea/workflows/demo.yaml`.
+    - Define a basic test pipeline:YAML
+        
+        # 
+        
+        ```
+        name: Gitea CI/CD
+        on: [push]
+        jobs:
+          test-build:
+            runs-on: ubuntu-latest
+            steps:
+              - name: Check out repository
+                uses: actions/checkout@v4
+              - name: Build Application
+                run: |
+                  echo "Compiling and testing..."
+                  # Add your test or build commands here
+        ```
+        
+
+### Phase 5: End-to-End Integration Test
+
+- [ ]  **Task 5.1: Push a Commit**
+    - Run `git push` from your local machine to the Gitea instance on Lightsail.
+    - Verify the sequence:
+        1. Gitea (on Lightsail) displays a pending job badge.
+        2. Webhook triggers on Lightsail (same VM as Gitea).
+        3. Modal sandbox boots up.
+        4. The sandbox logs into Gitea (via public Lightsail URL), processes the workflow, and turns off.
+        5. Gitea UI (on Lightsail) updates to green (Success) or red (Failure).

workspace/sprint_1_2/CODEBASE/deps/spec/secrets_management_guideline.md

@@ -0,0 +1,211 @@
+# Secrets Management Guideline
+
+This document defines how to handle secrets, environment variables, and sensitive configuration across the PILOT project CI/CD pipeline and infrastructure.
+
+## Principles
+
+1. **Never commit secrets to version control** — Use `.gitignore` and secret stores
+2. **Least-privilege access** — Each component gets only the secrets it needs
+3. **Environment isolation** — Separate secrets for local, staging, and production
+4. **Auditability** — Track secret creation, rotation, and access
+
+## Secret Storage Locations
+
+### 1. Gitea Repository Secrets (CI/CD Workflows)
+
+Store secrets used by GitHub Actions / Gitea Actions workflows in Gitea's built-in secrets manager.
+
+**Access path:** Gitea UI → Repository Settings → Secrets → Actions
+
+```bash
+# Repository-level secrets (per project)
+AWS_ACCESS_KEY_ID
+AWS_SECRET_ACCESS_KEY
+MODAL_TOKEN_ID
+MODAL_TOKEN_SECRET
+POSTGRES_PASSWORD
+GITEA_WEBHOOK_SECRET
+CADDY_API_TOKEN
+```
+
+**Access in workflow:**
+```yaml
+env:
+  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+  MODAL_TOKEN_ID: ${{ secrets.MODAL_TOKEN_ID }}
+  MODAL_TOKEN_SECRET: ${{ secrets.MODAL_TOKEN_SECRET }}
+```
+
+### 2. Modal Secrets (Serverless Workers)
+
+Store Modal-specific configuration in Modal's secrets system.
+
+**Create secret:**
+```bash
+modal secret create gitea-runner-secrets \
+  MODAL_KEY=your-modal-key \
+  MODAL_SECRET=your-modal-secret
+```
+
+**Use in deploy_runner.py:**
+```python
+import os
+import modal
+
+app = modal.App("gitea-runner")
+secret = modal.Secret.from_name("gitea-runner-secrets")
+
+@app.function(secret=secret)
+def run_runner():
+    modal_key = os.environ["MODAL_KEY"]
+    modal_secret = os.environ["MODAL_SECRET"]
+```
+
+### 3. AWS Secrets Manager (Production Infrastructure)
+
+Store AWS credentials and infrastructure secrets in AWS Secrets Manager.
+
+**Create secret:**
+```bash
+aws secretsmanager create-secret \
+  --name gitea/prod/aws-credentials \
+  --secret-string '{"access_key":"...","secret_key":"..."}'
+```
+
+**Retrieve secret:**
+```bash
+aws secretsmanager get-secret-value \
+  --secret-id gitea/prod/aws-credentials \
+  --query SecretString --output text
+```
+
+**Use in Python (boto3):**
+```python
+import boto3
+import json
+
+client = boto3.client('secretsmanager')
+secret = client.get_secret_value(SecretId='gitea/prod/aws-credentials')
+credentials = json.loads(secret['SecretString'])
+```
+
+### 4. Local Development (.env)
+
+Use `.env` files for local development with strict `.gitignore` rules.
+
+```bash
+# .env (NEVER commit this file)
+AWS_ACCESS_KEY_ID=local-dev-key
+AWS_SECRET_ACCESS_KEY=local-dev-secret
+AWS_DEFAULT_REGION=us-east-1
+HF_TOKEN=hf-local-token
+MODAL_KEY=local-modal-key
+MODAL_SECRET=local-modal-secret
+FIGMA_ACCESS_TOKEN=local-figma-token
+SUPABASE_URL=http://localhost:54321
+SUPABASE_PUBLISHABLE_KEY=local-supabase-key
+SUPABASE_DB_URL=postgresql://postgres:postgres@localhost:54322/postgres
+SUPABASE_SERVICE_ROLE_KEY=local-supabase-service-key
+EXA_API_KEY=local-exa-key
+GITHUB_PAT=local-github-pat
+GITHUB_GITEA_CLIENT_SECRET=local-gitea-client-secret
+POSTGRES_PASSWORD=local-password
+GITEA_URL=http://localhost:3000
+GITEA_WEBHOOK_SECRET=local-webhook-secret
+```
+
+```bash
+# .gitignore
+.env
+.env.local
+.env.*.local
+secrets/
+*.pem
+*.key
+```
+
+## Gitea Workflow Secrets Template
+
+Use this template to reference secrets in `.gitea/workflows/*.yml` files:
+
+```yaml
+env:
+  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+  AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
+  HF_TOKEN: ${{ secrets.HF_TOKEN }}
+  MODAL_KEY: ${{ secrets.MODAL_KEY }}
+  MODAL_SECRET: ${{ secrets.MODAL_SECRET }}
+  FIGMA_ACCESS_TOKEN: ${{ secrets.FIGMA_ACCESS_TOKEN }}
+  SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
+  SUPABASE_PUBLISHABLE_KEY: ${{ secrets.SUPABASE_PUBLISHABLE_KEY }}
+  SUPABASE_DB_URL: ${{ secrets.SUPABASE_DB_URL }}
+  SUPABASE_SERVICE_ROLE_KEY: ${{ secrets.SUPABASE_SERVICE_ROLE_KEY }}
+  EXA_API_KEY: ${{ secrets.EXA_API_KEY }}
+  GITHUB_PAT: ${{ secrets.GITHUB_PAT }}
+  GITHUB_GITEA_CLIENT_SECRET: ${{ secrets.GITHUB_GITEA_CLIENT_SECRET }}
+```
+
+All secrets must be registered in Gitea under **Repository Settings → Secrets → Actions** before the workflow runs.
+
+## Secret Reference Matrix
+
+| Secret | Local Dev | Gitea CI/CD | Modal Worker | AWS Infra |
+|--------|-----------|-------------|--------------|-----------|
+| AWS_ACCESS_KEY_ID | .env | Gitea Secret | — | Secrets Manager |
+| AWS_SECRET_ACCESS_KEY | .env | Gitea Secret | — | Secrets Manager |
+| AWS_DEFAULT_REGION | .env | Gitea Secret | — | — |
+| HF_TOKEN | .env | Gitea Secret | — | — |
+| MODAL_KEY | .env | Gitea Secret | Modal Secret | — |
+| MODAL_SECRET | .env | Gitea Secret | Modal Secret | — |
+| FIGMA_ACCESS_TOKEN | .env | Gitea Secret | — | — |
+| SUPABASE_URL | .env | Gitea Secret | — | — |
+| SUPABASE_PUBLISHABLE_KEY | .env | Gitea Secret | — | — |
+| SUPABASE_DB_URL | .env | Gitea Secret | — | Secrets Manager |
+| SUPABASE_SERVICE_ROLE_KEY | .env | Gitea Secret | — | — |
+| EXA_API_KEY | .env | Gitea Secret | — | — |
+| GITHUB_PAT | .env | Gitea Secret | — | — |
+| GITHUB_GITEA_CLIENT_SECRET | .env | Gitea Secret | — | — |
+| POSTGRES_PASSWORD | .env | Gitea Secret | — | Secrets Manager |
+| GITEA_WEBHOOK_SECRET | .env | Gitea Secret | — | — |
+| CADDY_API_TOKEN | .env | Gitea Secret | — | — |
+
+## Security Best Practices
+
+1. **Rotate secrets regularly** — Especially AWS keys and tokens (every 90 days)
+2. **Use IAM roles** — Prefer IAM roles over hardcoded AWS keys in production
+3. **Encrypt at rest** — Use AWS KMS for Secrets Manager encryption
+4. **Audit access** — Enable CloudTrail for AWS Secrets Manager, Gitea audit logs
+5. **Pre-commit hooks** — Use `git-secrets` or `pre-commit` to block secret commits
+6. **Separate environments** — Use different secret namespaces for dev/staging/prod
+7. **Minimal scope** — AWS keys should have only required permissions
+
+## Pre-commit Hook Setup (Optional)
+
+Install `git-secrets` to prevent committing secrets:
+
+```bash
+# Install git-secrets
+git secrets --install
+
+# Register common patterns
+git secrets --register-aws
+git secrets --add 'GITEA_TOKEN'
+git secrets --add 'MODAL_TOKEN'
+```
+
+## Emergency: Secret Exposure
+
+If a secret is accidentally committed:
+
+1. **Rotate immediately** — Generate a new secret value
+2. **Revoke the old secret** — Invalidate the exposed credential
+3. **Audit access logs** — Check for unauthorized usage
+4. **Update all references** — Deploy the new secret to all environments
+5. **Force push cleanup** — Use `git filter-branch` or `BFG` to remove from history
+
+## Related Files
+
+- `build_server_plan.md` — CI/CD pipeline architecture
+- `CICD_architecture.png` — Architecture diagram