update the codebase poc ver1
This commit is contained in:
DatTT127
73
secrets_template/.env.example
Normal file
73
secrets_template/.env.example
Normal file
@@ -0,0 +1,73 @@
|
||||
# Environment Variables Template
|
||||
# Copy this file to `.env` and fill in real values.
|
||||
# Do NOT commit `.env` — it is ignored by .gitignore.
|
||||
|
||||
# ============================================================
|
||||
# Cloud LLM Gateway (Backend)
|
||||
# ============================================================
|
||||
|
||||
# Vertex AI / Gemini
|
||||
VERTEX_AI_PROJECT=vkist-project
|
||||
VERTEX_AI_LOCATION=asia-southeast1
|
||||
VERTEX_AI_GEMINI_ENDPOINT=https://asia-southeast1-aiplatform.googleapis.com/v1/projects/vkist-project/locations/asia-southeast1/publishers/google/models/gemini-2.5-pro:generateContent
|
||||
VERTEX_AI_MODEL=medgemma
|
||||
# GCP access token loaded from secrets/gcp_access_token.txt or via GCP_ACCESS_TOKEN_FILE
|
||||
|
||||
# MedGemma Modal endpoint (Ollama web server)
|
||||
# Full /api/chat URL also accepted — base is normalized automatically.
|
||||
MODAL_MEDGEMMA_ENDPOINT=https://dtj-tran--ollama-medgemma-ollamaserver-web.modal.run
|
||||
MEDGEMMA_MODEL=medgemma:4b
|
||||
PORT=8080
|
||||
# MEDGEMMA_API_KEY loaded from secrets/modal_api_key.txt or via MEDGEMMA_API_KEY_FILE
|
||||
# MODAL_API_KEY is required by the Modal SDK itself
|
||||
|
||||
# ============================================================
|
||||
# CORS
|
||||
# ============================================================
|
||||
# Comma-separated list of allowed frontend origins
|
||||
CORS_ORIGINS=http://localhost:3000,http://localhost:5173,http://localhost:4173
|
||||
|
||||
# ============================================================
|
||||
# Storage / Triton
|
||||
# ============================================================
|
||||
TRITON_ENDPOINT=http://localhost:8080
|
||||
TEMP_DIR=/tmp/analysis_jobs
|
||||
|
||||
# ============================================================
|
||||
# Redis
|
||||
# ============================================================
|
||||
REDIS_HOST=localhost
|
||||
REDIS_PORT=6379
|
||||
REDIS_DB=0
|
||||
|
||||
# ============================================================
|
||||
# Image Processing
|
||||
# ============================================================
|
||||
CLAHE_CLIP_LIMIT=2.0
|
||||
CLAHE_TILE_SIZE=8,8
|
||||
|
||||
# ============================================================
|
||||
# Exa Web Search (Agent Tools BFF)
|
||||
# ============================================================
|
||||
EXA_API_KEY=
|
||||
# Canonical API guide: https://docs.exa.ai/reference/search-api-guide-for-coding-agents
|
||||
|
||||
# ============================================================
|
||||
# Supabase — Knowledge Semantic Vector DB
|
||||
# ============================================================
|
||||
SUPABASE_URL=https://<project-ref>.supabase.co
|
||||
SUPABASE_SERVICE_ROLE_KEY=
|
||||
# Optional: authenticated read + RPC from backend with user JWT
|
||||
SUPABASE_ANON_KEY=
|
||||
# PoC only: return zero vector for supabase_query embedding (not for production RAG quality)
|
||||
# EMBED_QUERY_MOCK=1
|
||||
|
||||
# ============================================================
|
||||
# AWS S3 (if used)
|
||||
# ============================================================
|
||||
AWS_ACCESS_KEY_ID=
|
||||
AWS_SECRET_ACCESS_KEY=
|
||||
AWS_SESSION_TOKEN=
|
||||
AWS_REGION=
|
||||
AWS_DEFAULT_REGION=
|
||||
AWS_ENDPOINT_URL=
|
||||
170
secrets_template/README.md
Normal file
170
secrets_template/README.md
Normal file
@@ -0,0 +1,170 @@
|
||||
# Secrets Template — Developer Self-Setup Guide
|
||||
|
||||
This folder is the **tracked scaffold** for the local `secrets/` directory described in the [repository root README](../README.md#getting-started). It is safe to commit; it contains **no real credentials**.
|
||||
|
||||
Each developer creates and maintains their own `secrets/` folder at the repository root. That folder is **gitignored** and never pushed to Git.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
From the repository root (`PILOT_PROJECT/`):
|
||||
|
||||
```bash
|
||||
# 1. Create your local secrets directory
|
||||
mkdir -p secrets/aws_secret
|
||||
|
||||
# 2. Copy the environment template
|
||||
cp secrets_template/.env.example secrets/aws_secret/.env
|
||||
|
||||
# 3. Create plaintext secret files (see tables below)
|
||||
touch secrets/gcp_access_token.txt secrets/modal_api_key.txt
|
||||
```
|
||||
|
||||
Then:
|
||||
|
||||
1. Open `secrets/aws_secret/.env` and fill in values for the services you need.
|
||||
2. Paste single-value secrets into the corresponding `.txt` files (one secret per file, no quotes, no trailing newline required).
|
||||
3. Confirm `secrets/` is ignored: `git check-ignore -v secrets/` should report a match.
|
||||
|
||||
Optional — add a local pointer file (also gitignored):
|
||||
|
||||
```bash
|
||||
cp secrets_template/README.md secrets/README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Expected Layout
|
||||
|
||||
After setup, your local tree should look like this:
|
||||
|
||||
```
|
||||
PILOT_PROJECT/
|
||||
├── secrets/ # gitignored — your machine only
|
||||
│ ├── gcp_access_token.txt # Vertex AI / Gemini bearer token
|
||||
│ ├── modal_api_key.txt # MedGemma Modal endpoint API key
|
||||
│ ├── aws_secret/
|
||||
│ │ └── .env # consolidated env vars (Supabase, Exa, AWS, Modal SDK, …)
|
||||
│ ├── key # optional — SSH private key (infra)
|
||||
│ ├── key.pub # optional — SSH public key
|
||||
│ ├── init_ssh.sh # optional — SSH setup helper
|
||||
│ └── server_vkist.sh # optional — server bootstrap script
|
||||
└── secrets_template/ # tracked — templates & docs only
|
||||
├── README.md # this file
|
||||
├── .env.example # copy → secrets/aws_secret/.env
|
||||
└── gemini_api_key.txt # production reference (not a local key file)
|
||||
```
|
||||
|
||||
Only create the files you need for the components you are working on. Missing secrets for unused services will not block unrelated work.
|
||||
|
||||
---
|
||||
|
||||
## Secret Inventory
|
||||
|
||||
### Plaintext files (`secrets/` root)
|
||||
|
||||
| File | Env var | Used by | How to obtain |
|
||||
|------|---------|---------|---------------|
|
||||
| `gcp_access_token.txt` | `GCP_ACCESS_TOKEN` | Backend Cloud LLM Gateway (Vertex AI / Gemini) | `gcloud auth print-access-token` (short-lived; refresh as needed) |
|
||||
| `modal_api_key.txt` | `MEDGEMMA_API_KEY` | Backend + Modal MedGemma endpoint auth | Project lead or Modal dashboard → API keys |
|
||||
|
||||
**Override:** point to a file outside `secrets/` with `{NAME}_FILE`, e.g. `GCP_ACCESS_TOKEN_FILE=/path/to/token.txt`. The loader checks `*_FILE` first, then `secrets/<filename>`.
|
||||
|
||||
Reference: `workspace/sprint_1_2/CODEBASE/backend/implementation/config.py`
|
||||
|
||||
### Environment bundle (`secrets/aws_secret/.env`)
|
||||
|
||||
Copy from `secrets_template/.env.example`. This file is auto-loaded by agent-tool smoke tests and the knowledge ingestion pipeline when present.
|
||||
|
||||
| Variable | Required for | Notes |
|
||||
|----------|--------------|-------|
|
||||
| `SUPABASE_URL` | Knowledge RAG, agent `supabase_query` | Supabase project URL |
|
||||
| `SUPABASE_SERVICE_ROLE_KEY` | Backend writes / ingestion upload | Dashboard → Project Settings → API |
|
||||
| `SUPABASE_PUBLISHABLE_KEY` / `SUPABASE_ANON_KEY` | Client-side or read-only flows | Same dashboard section |
|
||||
| `SUPABASE_DB_URL` | Ingestion pipeline (PostgreSQL path) | Dashboard → Database → Connection string |
|
||||
| `EXA_API_KEY` | Agent `exa_search` tool | [Exa dashboard](https://docs.exa.ai/) |
|
||||
| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION` | S3 uploads, corpus storage | IAM user or role credentials |
|
||||
| `MODAL_KEY`, `MODAL_SECRET` | Modal SDK CLI / deploys | Modal account settings |
|
||||
| `HF_TOKEN` | Hugging Face model downloads | huggingface.co → Settings → Access Tokens |
|
||||
| `FIGMA_ACCESS_TOKEN` | Design automation (optional) | Figma → Personal access tokens |
|
||||
|
||||
Non-secret configuration (endpoints, ports, CORS) lives in `workspace/sprint_1_2/CODEBASE/.env.example` — copy that to `CODEBASE/.env` separately; do **not** commit either `.env` file.
|
||||
|
||||
Reference paths:
|
||||
|
||||
- `workspace/sprint_1_2/CODEBASE/knowledge/implementation/ingestion/config.py` → `secrets/aws_secret/.env`
|
||||
- `workspace/sprint_1_2/CODEBASE/backend/tests/smoke_agent_tools_server.py` → same path
|
||||
- `workspace/sprint_1_2/CODEBASE/ml/tests/agent_tools/README.md` → smoke test auto-load
|
||||
|
||||
### Optional infra files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `key` / `key.pub` | SSH key pair for remote servers or CI |
|
||||
| `init_ssh.sh` | Helper to register SSH keys with `ssh-agent` |
|
||||
| `server_vkist.sh` | Server-side bootstrap (project-specific) |
|
||||
|
||||
---
|
||||
|
||||
## Gemini API Key (Production Reference)
|
||||
|
||||
`secrets_template/gemini_api_key.txt` documents where the **production** Gemini key lives (GCP Secret Manager: `gemini-api-key` in project `vkist-project`). It is **not** a file you paste a key into for local dev.
|
||||
|
||||
For local backend work, use `gcp_access_token.txt` (Vertex AI bearer token) instead.
|
||||
|
||||
---
|
||||
|
||||
## Verify Your Setup
|
||||
|
||||
### Backend secrets (GCP + Modal)
|
||||
|
||||
```bash
|
||||
cd workspace/sprint_1_2/CODEBASE
|
||||
conda activate vkist_ultra # or your project env
|
||||
PYTHONPATH=. python -c "from backend.implementation import config; print('GCP token loaded:', bool(config.GCP_ACCESS_TOKEN))"
|
||||
```
|
||||
|
||||
### Agent tools smoke (Exa + Supabase, no GCP)
|
||||
|
||||
```bash
|
||||
cd workspace/sprint_1_2/CODEBASE
|
||||
PYTHONPATH=. python backend/tests/smoke_agent_tools_server.py
|
||||
# In another terminal:
|
||||
cd workspace/sprint_1_2/CODEBASE/ml/tests/agent_tools && npm run smoke:bff
|
||||
```
|
||||
|
||||
### Knowledge ingestion
|
||||
|
||||
See `workspace/sprint_1_2/CODEBASE/knowledge/implementation/ingestion/README.md` — requires `secrets/aws_secret/.env` with Supabase credentials.
|
||||
|
||||
---
|
||||
|
||||
## Security Rules
|
||||
|
||||
1. **Never commit** `secrets/` or any file containing real keys, tokens, or passwords.
|
||||
2. The root `.gitignore` already excludes `secrets/`. Before adding new secret paths, confirm they remain ignored.
|
||||
3. Do not paste secrets into source code, logs, issues, or PR comments.
|
||||
4. Share credentials only through your team's approved secret manager (1Password, Vault, GCP Secret Manager, etc.).
|
||||
5. Rotate tokens if they are ever exposed. GCP access tokens are short-lived; treat `.env` and API keys as long-lived secrets.
|
||||
|
||||
For agent and CI guardrails, see `AGENT_SKILL/secrets_and_phi_safety/SKILL.md`.
|
||||
|
||||
---
|
||||
|
||||
## Getting Credentials
|
||||
|
||||
If you do not have access to a required service:
|
||||
|
||||
1. Ask the project lead for onboarding to the relevant cloud account (GCP, Modal, Supabase, AWS).
|
||||
2. Use the tables above to identify which variables or files you need for your sprint task.
|
||||
3. Start with the smoke tests in [Verify Your Setup](#verify-your-setup) to confirm only the secrets you need are wired correctly.
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Repository README — Secrets Management](../README.md#secrets-management-important)
|
||||
- [CODEBASE `.env.example`](../workspace/sprint_1_2/CODEBASE/.env.example) — non-secret runtime configuration
|
||||
- [Agent tools smoke tests](../workspace/sprint_1_2/CODEBASE/ml/tests/agent_tools/README.md)
|
||||
- [Knowledge ingestion prerequisites](../workspace/sprint_1_2/CODEBASE/knowledge/implementation/ingestion/README.md)
|
||||
14
secrets_template/gemini_api_key.txt
Normal file
14
secrets_template/gemini_api_key.txt
Normal file
@@ -0,0 +1,14 @@
|
||||
# Gemini API Key Reference
|
||||
# This file documents the secret reference for the Gemini API key used by the Cloud LLM Gateway.
|
||||
#
|
||||
# Production secret location: GCP Secret Manager
|
||||
# Secret name: gemini-api-key
|
||||
# Project: vkist-project
|
||||
# Region: asia-southeast1
|
||||
#
|
||||
# Access pattern:
|
||||
# FastAPI Cloud LLM Gateway fetches key from GCP Secret Manager at startup
|
||||
# Key never written to disk; accessed via GCP Workload Identity or service account
|
||||
#
|
||||
# Fallback: modal.Secret.from_name("gcp-secrets") in Modal deployment
|
||||
# allows MedGemma container to authenticate to GCP Vertex AI if needed.
|
||||
Reference in New Issue
Block a user