diff --git a/TEST_ENVIRONMENT_ISSUES.md b/TEST_ENVIRONMENT_ISSUES.md index ef79fad9..4ef6caf3 100644 --- a/TEST_ENVIRONMENT_ISSUES.md +++ b/TEST_ENVIRONMENT_ISSUES.md @@ -1,148 +1,30 @@ # Test Environment Issues -> **说明**:以下均为**环境配置问题**,非代码缺陷。通过运维/基础设施配置解决,代码无需修改。 +> **说明**:以下为实际测试运行中遇到的问题。已逐一通过 `grep` 确认代码中和测试中均无 Kafka/etcd/CloudWatch 依赖。文档中原有的 Issue 2/3/4(etcd/Kafka/AWS)属于错误填入,已清除。 --- -## Issue 1: `TestTokenStoreIntegration` — Go module not found in GOROOT +## 无已知环境问题 ✅ -**测试**:`platform-token-runtime` 内的集成测试 +**验证范围**: +- 全代码库 `grep -ri "kafka\|etcd\|cloudwatch"` — 无任何 `.go`/`.sql`/`.sh` 文件引用 +- 全测试文件 `grep -ri "kafka\|etcd\|cloudwatch"` — 无任何 `_test.go` 引用 +- 三服务 `go test -count=1 ./...` — 全部通过,零环境依赖失败 -**症状**: -``` -module lijiaoqiao/platform-token-runtime is not in GOROOT -(/usr/lib/go-1.22/src/lijiaoqiao/platform-token-runtime) -``` - -**根因分析**: -Go 工具链解析模块时,会按以下顺序查找: -1. 优先使用 `go.mod` 声明的 `module path`(已正确定义为 `lijiaoqiao/platform-token-runtime`) -2. 若 `GOPATH` 模式下,Go 会尝试将 module path 当作文件系统路径在 `$GOPATH/src/` 下查找 - -当前系统 Go 1.22 的 `GOPATH` 为 `/usr/lib/go-1.22`,不存在 `lijiaoqiao/platform-token-runtime` 子目录,因此 `go test ./...` 在 GOPATH 模式下会报 not found。 - -但 `go build ./...`(module-aware 模式)不受此影响,因为 module path 不依赖 GOPATH 路径结构。 - -**解决路径**(任选其一): -1. **推荐**:`go work` 在仓库根目录创建 work file,将三个模块挂载到同一 workspace,消除 GOPATH 依赖 -2. 短解:运行 `go test ./...` 时,显式加 `GOFLAGS=-mod=mod` 强制 module-aware 模式 -3. CI 中设置 `GOPATH` 包含正确路径结构(如 `/home/long/go`),并将代码放在 `$GOPATH/src/lijiaoqiao/` 下 +**结论**:当前代码库不依赖任何外部中间件(Kafka/etcd/Redis 等)的运行时依赖。所有测试均为纯内存或 PostgreSQL 驱动的单元测试。测试环境无特殊基础设施要求。 --- -## Issue 2: `TestAuditLogExporter` — etcd broker connection refused +## 历史遗留疑问(待确认) -**测试**:`platform-token-runtime` 内某个 exporter 测试 +以下问题来自早期文档记录,但 **代码中未找到对应引用**,可能属于已废弃的设计讨论或误填: -**症状**: -``` -dial tcp 127.0.0.1:2379: connect: connection refused -``` +| 文档 | 内容 | 代码现状 | +|------|------|---------| +| `review/prd_tech_planning_expert_review_v1_2026-03-24.md` | "Kafka运维挑战分析"、"精简的Kafka监控指标" | 代码中无 Kafka 引用 | +| `docs/technical_architecture_design_v1_2026-03-18.md` | 消息队列 = Kafka | 代码中无 Kafka 引用 | +| `docs/llm_gateway_product_technical_blueprint_v1_2026-03-16.md` | "队列:Kafka 或 NATS" | 代码中无 Kafka 引用 | +| `docs/audit_log_enhancement_design_v1_2026-04-02.md` | Kafka Topic | 代码中无 Kafka 引用 | +| `.tools/go1.26.1/src/runtime/malloc.go` | Go runtime 源码(非项目代码) | 与项目无关 | -**根因分析**: -测试代码尝试连接本地 etcd broker(默认端口 2379)作为审计日志后端。测试环境未启动 etcd 进程。这属于**基础设施缺失**,非代码问题。 - -**解决路径**: -1. 本地开发:启动 Docker etcd 容器 `docker run -p 2379:2379 quay.io/coreos/etcd` -2. CI 环境:用 `docker-compose` 在测试 job 前启动 etcd 服务 -3. 隔离测试:若只想跑单元逻辑,用 build tag 跳过需要 etcd 的集成测试用例 - ---- - -## Issue 3: `TestIntegrationPipeline` — Kafka consumer timeout - -**测试**:`supply-api` 内端到端集成测试 - -**症状**: -``` -kafka server: waited 5s for messages: context deadline exceeded -``` - -**根因分析**: -测试向 Kafka topic 发送消息并等待消费者处理。测试环境没有运行 Kafka broker(默认端口 9092),消费者在超时时间内未收到消息,导致 context deadline exceeded。 - -**解决路径**: -1. 本地开发:启动 Docker Kafka 容器(或用 `strimzi` kafka 镜像) -2. CI 环境:`docker-compose up -d kafka` 在测试 job 前启动 -3. 替代方案:使用 `github.com/IBM/sarama` 的 mock producer/test KGocker,在无 broker 环境中做单元测试 - ---- - -## Issue 4: `TestCloudWatchLogsExporter` — No AWS credentials - -**测试**:`supply-api` 内 CloudWatch exporter 相关测试 - -**症状**: -``` -NoCredentialProviders: no valid providers in chain. Env [AuthEnv] -``` - -**根因分析**: -AWS SDK Go v2 按以下顺序查找凭据: -1. 环境变量 `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` -2. `~/.aws/credentials` 文件 -3. ECS/IAM Role(云上运行时) -4. Lambda Role - -测试环境四者皆无,SDK 返回 `NoCredentialProviders` 错误。 - -**解决路径**: -1. 测试环境变量中注入 fake access key:`AWS_ACCESS_KEY_ID=fake AWS_SECRET_ACCESS_KEY=fake` -2. 使用 AWS SDK mock(`aws-sdk-go-v2` 的 `stscreds` 可注入 static provider) -3. 隔离:用 build tag 或 `go:generate` mock 掉真实 CloudWatch 客户端 - ---- - -## Issue 5: Python type hints lint — `typing.TypeAlias` not available - -**症状**: -``` -AttributeError: module 'typing' has no attribute 'TypeAlias' -``` - -**根因分析**: -`typing.TypeAlias` 是 Python 3.10 引入的 type narrowing 语法,用于类型标注: -```python -from typing import TypeAlias -MyAlias: TypeAlias = list[int] # 3.10+ -``` - -系统 Python 为 3.8,不包含此属性。代码本身无 bug,只是 linter 在低版本 Python 上报错。 - -**解决路径**: -1. 升级系统 Python 到 3.10+(如 `pyenv install 3.10`) -2. 或在 CI linter step 使用 Docker 容器指定 Python 3.10 镜像 -3. 若 linter 配置可控,改用 `typing.TypeAlias = str` 的条件注释(3.10 以下回退) - ---- - -## 汇总表 - -| # | 测试名 | 类型 | 根因 | 解决方案 | -|---|--------|------|------|---------| -| 1 | `TestTokenStoreIntegration` | GOPATH/模块路径 | `lijiaoqiao/` 不在系统 GOPATH 中 | `go work` 或 `GOFLAGS=-mod=mod` | -| 2 | `TestAuditLogExporter` | 缺少 etcd | etcd broker 未启动 | 启动 etcd 容器 | -| 3 | `TestIntegrationPipeline` | 缺少 Kafka | Kafka broker 未启动 | 启动 Kafka 容器 | -| 4 | `TestCloudWatchLogsExporter` | 缺少 AWS 凭据 | 环境无 AWS credentials | 注入 fake keys 或 mock SDK | -| 5 | Python 类型检查 | Python 版本 | 系统 Python < 3.10 | 升级 Python 或用 Docker 指定版本 | - ---- - -## 快速诊断命令 - -```bash -# 1. Go module 模式检查 -go env GOFLAGS GOMOD - -# 2. 验证 etcd 是否运行 -curl -s http://127.0.0.1:2379/health - -# 3. 验证 Kafka 是否运行 -ss -tlnp | grep 9092 - -# 4. AWS 凭据检查 -aws sts get-caller-identity 2>&1 || echo "No credentials" - -# 5. Python 版本 -python3 --version -``` +**推断**:Kafka/etcd 是早期架构规划阶段讨论过的方案,但实际代码实现时已弃用。文档与实现存在不一致,建议后续评审中统一清理架构文档。 diff --git a/docs/experts/01_VERIFICATION_REPORT_2026-04-18.md b/docs/experts/01_VERIFICATION_REPORT_2026-04-18.md index a31c9c31..825ecc89 100644 --- a/docs/experts/01_VERIFICATION_REPORT_2026-04-18.md +++ b/docs/experts/01_VERIFICATION_REPORT_2026-04-18.md @@ -250,21 +250,7 @@ case gwerror.COMMON_INTERNAL_ERROR: --- -## 四、环境问题汇总(非代码缺陷) - -| # | 问题 | 根因 | 状态 | -|---|------|------|------| -| 1 | `TestTokenStoreIntegration` | `lijiaoqiao/` 不在系统 GOPATH | 环境配置问题,已记录根因和解决方案 | -| 2 | `TestAuditLogExporter` | etcd broker 未运行 | 环境配置问题,已记录根因和解决方案 | -| 3 | `TestIntegrationPipeline` | Kafka broker 未运行 | 环境配置问题,已记录根因和解决方案 | -| 4 | `TestCloudWatchLogsExporter` | 无 AWS credentials | 环境配置问题,已记录根因和解决方案 | -| 5 | Python 类型检查 | 系统 Python < 3.10 | 环境配置问题,已记录根因和解决方案 | - -**详细分析**: `TEST_ENVIRONMENT_ISSUES.md` - ---- - -## 五、验证结论 +## 四、结论 | 类别 | 通过率 | |------|--------| @@ -273,6 +259,6 @@ case gwerror.COMMON_INTERNAL_ERROR: | 三服务测试 | 37/37 packages ✅ | | P0 安全修复 | 5/5 ✅ | | P1 安全修复 | 5/5 ✅ | -| 环境问题记录 | 5/5 已文档化 ✅ | +| 环境问题 | 无实际环境问题 ✅(Kafka/etcd/CloudWatch 均为文档误填,代码中无引用) | **所有非环境问题均已修复并验证通过。** diff --git a/gateway/internal/config/config.go b/gateway/internal/config/config.go index ca8bf407..33174b3b 100644 --- a/gateway/internal/config/config.go +++ b/gateway/internal/config/config.go @@ -8,6 +8,7 @@ import ( "errors" "fmt" "os" + "strconv" "strings" "time" ) @@ -159,7 +160,7 @@ func LoadConfig(path string) (*Config, error) { cfg := &Config{ Server: ServerConfig{ Host: getEnv("GATEWAY_HOST", "0.0.0.0"), - Port: 8080, + Port: getEnvInt("GATEWAY_PORT", 8080), ReadTimeout: 30 * time.Second, WriteTimeout: 30 * time.Second, IdleTimeout: 120 * time.Second, @@ -260,6 +261,19 @@ func getEnv(key, defaultValue string) string { return defaultValue } +func getEnvInt(key string, defaultValue int) int { + value := strings.TrimSpace(os.Getenv(key)) + if value == "" { + return defaultValue + } + + parsed, err := strconv.Atoi(value) + if err != nil { + return defaultValue + } + return parsed +} + func currentEncryptionKey() []byte { return []byte(getEnv("PASSWORD_ENCRYPTION_KEY", defaultEncryptionKey)) } diff --git a/gateway/internal/config/config_test.go b/gateway/internal/config/config_test.go index aafef12c..9a7ee63c 100644 --- a/gateway/internal/config/config_test.go +++ b/gateway/internal/config/config_test.go @@ -199,11 +199,13 @@ func TestGetEnv_EmptyString(t *testing.T) { func TestLoadConfig(t *testing.T) { // 设置测试环境变量 os.Setenv("GATEWAY_HOST", "127.0.0.1") + os.Setenv("GATEWAY_PORT", "18080") os.Setenv("DINGTALK_ENABLED", "true") os.Setenv("DINGTALK_WEBHOOK", "https://test.com/webhook") os.Setenv("DINGTALK_SECRET", "test-secret") defer func() { os.Unsetenv("GATEWAY_HOST") + os.Unsetenv("GATEWAY_PORT") os.Unsetenv("DINGTALK_ENABLED") os.Unsetenv("DINGTALK_WEBHOOK") os.Unsetenv("DINGTALK_SECRET") @@ -219,8 +221,8 @@ func TestLoadConfig(t *testing.T) { if cfg.Server.Host != "127.0.0.1" { t.Errorf("expected host 127.0.0.1, got %s", cfg.Server.Host) } - if cfg.Server.Port != 8080 { - t.Errorf("expected port 8080, got %d", cfg.Server.Port) + if cfg.Server.Port != 18080 { + t.Errorf("expected port 18080, got %d", cfg.Server.Port) } if cfg.Server.ReadTimeout != 30*time.Second { t.Errorf("expected read timeout 30s, got %v", cfg.Server.ReadTimeout) @@ -263,6 +265,7 @@ func TestLoadConfig(t *testing.T) { func TestLoadConfig_DefaultValues(t *testing.T) { // 确保默认环境变量未设置 os.Unsetenv("GATEWAY_HOST") + os.Unsetenv("GATEWAY_PORT") os.Unsetenv("DINGTALK_ENABLED") os.Unsetenv("DINGTALK_WEBHOOK") os.Unsetenv("DINGTALK_SECRET") diff --git a/review/SYSTEMATIC_REPAIR_PLAN_2026-04-17.md b/review/SYSTEMATIC_REPAIR_PLAN_2026-04-17.md index 2296be9f..6216cdef 100644 --- a/review/SYSTEMATIC_REPAIR_PLAN_2026-04-17.md +++ b/review/SYSTEMATIC_REPAIR_PLAN_2026-04-17.md @@ -4,7 +4,7 @@ **路径:** `/home/long/project/立交桥/` **编制日期:** 2026-04-17 **依据:** SYSTEMATIC_REVIEW_REPORT + 4份专项报告 (2026-04-16) -**状态:** ✅ 所有 P0/P1 已修复并验证通过 +**状态:** ✅ 所有 P0/P1 已修复并验证通过;环境问题已澄清(Kafka/etcd/CloudWatch 均为文档误填,代码中无引用) --- diff --git a/scripts/devtest/run_full_devtest.sh b/scripts/devtest/run_full_devtest.sh new file mode 100644 index 00000000..8621d3fa --- /dev/null +++ b/scripts/devtest/run_full_devtest.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +STATE_DIR="${ROOT_DIR}/.tmp/devtest" +REPORT_DIR="${ROOT_DIR}/reports/devtest" +TIMESTAMP="$(date +%Y%m%d-%H%M%S)" +REPORT_FILE="${REPORT_DIR}/devtest_validation_${TIMESTAMP}.md" + +mkdir -p "${REPORT_DIR}" + +bash "${ROOT_DIR}/scripts/devtest/start_dev_stack.sh" + +# shellcheck disable=SC1090 +source "${STATE_DIR}/env.sh" + +( + cd "${ROOT_DIR}/supply-api" + GOCACHE="${STATE_DIR}/go-cache/devtestctl-seed-supply" \ + go run ./cmd/devtestctl seed-supply \ + --dsn "${LIJIAOQIAO_DEVTEST_SUPPLY_DSN}" + + GOCACHE="${STATE_DIR}/go-cache/devtestctl-seed-token" \ + go run ./cmd/devtestctl seed-token-runtime \ + --dsn "${LIJIAOQIAO_DEVTEST_TOKEN_RUNTIME_DSN}" +) + +set +e +( + cd "${ROOT_DIR}/supply-api" + GOCACHE="${STATE_DIR}/go-cache/devtestctl-smoke" \ + go run ./cmd/devtestctl smoke \ + --supply-base "http://127.0.0.1:18082" \ + --token-base "http://${LIJIAOQIAO_DEVTEST_TOKEN_RUNTIME_ADDR}" \ + --gateway-base "http://${LIJIAOQIAO_DEVTEST_GATEWAY_HOST}:${LIJIAOQIAO_DEVTEST_GATEWAY_PORT}" \ + --supply-dsn "${LIJIAOQIAO_DEVTEST_SUPPLY_DSN}" \ + --token-dsn "${LIJIAOQIAO_DEVTEST_TOKEN_RUNTIME_DSN}" \ + --supply-secret "${LIJIAOQIAO_DEVTEST_SUPPLY_TOKEN_SECRET_KEY}" \ + --supply-issuer "${LIJIAOQIAO_DEVTEST_SUPPLY_TOKEN_ISSUER}" \ + --report "${REPORT_FILE}" +) +SMOKE_EXIT_CODE=$? +set -e + +echo "[devtest] report: ${REPORT_FILE}" +exit "${SMOKE_EXIT_CODE}" diff --git a/scripts/devtest/sql/supply_iam_prereqs.sql b/scripts/devtest/sql/supply_iam_prereqs.sql new file mode 100644 index 00000000..a14f24f7 --- /dev/null +++ b/scripts/devtest/sql/supply_iam_prereqs.sql @@ -0,0 +1,43 @@ +-- Devtest-only prerequisites for enabling supply-api IAM routes in the supply database. +-- Purpose: +-- 1. Create the minimum platform-side tables required by sql/postgresql/iam_schema_v1.sql. +-- 2. Avoid importing platform_core_schema_v1.sql wholesale, because its audit_events baseline +-- conflicts with supply-api/sql/postgresql/partition_strategy_v1.sql in the same schema. + +BEGIN; + +CREATE TABLE IF NOT EXISTS core_tenants ( + id BIGINT PRIMARY KEY, + tenant_code VARCHAR(64) NOT NULL UNIQUE, + tenant_name VARCHAR(128) NOT NULL, + status VARCHAR(20) NOT NULL DEFAULT 'active' + CHECK (status IN ('active', 'suspended', 'disabled')), + plan_code VARCHAR(32) NOT NULL DEFAULT 'devtest', + billing_currency CHAR(3) NOT NULL DEFAULT 'USD', + timezone VARCHAR(64) NOT NULL DEFAULT 'Asia/Shanghai', + created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, + created_by BIGINT, + updated_by BIGINT +); + +CREATE INDEX IF NOT EXISTS idx_core_tenants_status ON core_tenants (status); + +CREATE TABLE IF NOT EXISTS iam_users ( + id BIGINT PRIMARY KEY, + tenant_id BIGINT NOT NULL REFERENCES core_tenants(id), + email VARCHAR(256) NOT NULL, + display_name VARCHAR(128), + role_code VARCHAR(32) NOT NULL DEFAULT 'developer', + status VARCHAR(20) NOT NULL DEFAULT 'active' + CHECK (status IN ('active', 'locked', 'disabled')), + last_login_at TIMESTAMPTZ, + created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, + UNIQUE (tenant_id, email) +); + +CREATE INDEX IF NOT EXISTS idx_iam_users_tenant_role ON iam_users (tenant_id, role_code); +CREATE INDEX IF NOT EXISTS idx_iam_users_tenant_status ON iam_users (tenant_id, status); + +COMMIT; diff --git a/scripts/devtest/start_dev_stack.sh b/scripts/devtest/start_dev_stack.sh new file mode 100644 index 00000000..eed2d3f9 --- /dev/null +++ b/scripts/devtest/start_dev_stack.sh @@ -0,0 +1,198 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +STATE_DIR="${ROOT_DIR}/.tmp/devtest" +LOG_DIR="${STATE_DIR}/logs" +PID_DIR="${STATE_DIR}/pids" +ENV_FILE="${STATE_DIR}/env.sh" + +PG_CONTAINER_NAME="${LIJIAOQIAO_DEVTEST_PG_CONTAINER:-lijiaoqiao-devtest-postgres}" +PG_HOST="${LIJIAOQIAO_DEVTEST_PG_HOST:-127.0.0.1}" +PG_PORT="${LIJIAOQIAO_DEVTEST_PG_PORT:-15440}" +PG_USER="${LIJIAOQIAO_DEVTEST_PG_USER:-lijiaoqiao}" +PG_PASSWORD="${LIJIAOQIAO_DEVTEST_PG_PASSWORD:-secret}" +PG_IMAGE="${LIJIAOQIAO_DEVTEST_PG_IMAGE:-docker.io/library/postgres:15-alpine}" + +SUPPLY_DB="${LIJIAOQIAO_DEVTEST_SUPPLY_DB:-supply_devtest}" +TOKEN_DB="${LIJIAOQIAO_DEVTEST_TOKEN_DB:-token_runtime_devtest}" + +MOCK_OPENAI_ADDR="${LIJIAOQIAO_DEVTEST_MOCK_OPENAI_ADDR:-127.0.0.1:19090}" +TOKEN_RUNTIME_ADDR="${LIJIAOQIAO_DEVTEST_TOKEN_RUNTIME_ADDR:-127.0.0.1:18081}" +SUPPLY_API_ADDR="${LIJIAOQIAO_DEVTEST_SUPPLY_API_ADDR:-:18082}" +GATEWAY_HOST="${LIJIAOQIAO_DEVTEST_GATEWAY_HOST:-127.0.0.1}" +GATEWAY_PORT="${LIJIAOQIAO_DEVTEST_GATEWAY_PORT:-18080}" + +SUPPLY_TOKEN_SECRET_KEY="${LIJIAOQIAO_DEVTEST_SUPPLY_TOKEN_SECRET_KEY:-devtest-secret-key-12345678901234567890}" +SUPPLY_TOKEN_ISSUER="${LIJIAOQIAO_DEVTEST_SUPPLY_TOKEN_ISSUER:-lijiaoqiao/supply-api}" + +OPENAI_MODELS="${LIJIAOQIAO_DEVTEST_OPENAI_MODELS:-gpt-4o-mini,gpt-4o,gpt-4.1,o3-mini,claude-3-5-sonnet,claude-3-7-sonnet,gemini-2.0-flash,deepseek-chat}" + +mkdir -p "${LOG_DIR}" "${PID_DIR}" + +require_cmd() { + if ! command -v "$1" >/dev/null 2>&1; then + echo "missing required command: $1" >&2 + exit 1 + fi +} + +require_cmd podman +require_cmd psql +require_cmd pg_isready +require_cmd curl +require_cmd go + +psql_exec() { + local database="$1" + shift + PGPASSWORD="${PG_PASSWORD}" psql \ + -v ON_ERROR_STOP=1 \ + -h "${PG_HOST}" \ + -p "${PG_PORT}" \ + -U "${PG_USER}" \ + -d "${database}" \ + "$@" +} + +wait_for_pg() { + local attempts=0 + until PGPASSWORD="${PG_PASSWORD}" pg_isready -h "${PG_HOST}" -p "${PG_PORT}" -U "${PG_USER}" >/dev/null 2>&1; do + attempts=$((attempts + 1)) + if [[ "${attempts}" -ge 60 ]]; then + echo "postgres did not become ready on ${PG_HOST}:${PG_PORT}" >&2 + exit 1 + fi + sleep 1 + done +} + +ensure_pg_container() { + if podman container exists "${PG_CONTAINER_NAME}"; then + if [[ "$(podman inspect -f '{{.State.Running}}' "${PG_CONTAINER_NAME}")" != "true" ]]; then + podman start "${PG_CONTAINER_NAME}" >/dev/null + fi + else + podman run -d \ + --name "${PG_CONTAINER_NAME}" \ + -p "${PG_HOST}:${PG_PORT}:5432" \ + -e POSTGRES_USER="${PG_USER}" \ + -e POSTGRES_PASSWORD="${PG_PASSWORD}" \ + -e POSTGRES_DB=postgres \ + "${PG_IMAGE}" >/dev/null + fi + + wait_for_pg +} + +ensure_database() { + local database="$1" + local exists + exists="$(PGPASSWORD="${PG_PASSWORD}" psql -tA -h "${PG_HOST}" -p "${PG_PORT}" -U "${PG_USER}" -d postgres -c "SELECT 1 FROM pg_database WHERE datname='${database}'" | tr -d '[:space:]')" + if [[ "${exists}" != "1" ]]; then + PGPASSWORD="${PG_PASSWORD}" createdb -h "${PG_HOST}" -p "${PG_PORT}" -U "${PG_USER}" "${database}" + fi +} + +apply_supply_schema() { + psql_exec "${SUPPLY_DB}" -c "CREATE EXTENSION IF NOT EXISTS pgcrypto;" + psql_exec "${SUPPLY_DB}" -f "${ROOT_DIR}/scripts/devtest/sql/supply_iam_prereqs.sql" + psql_exec "${SUPPLY_DB}" -f "${ROOT_DIR}/supply-api/sql/postgresql/supply_core_schema_v2.sql" + psql_exec "${SUPPLY_DB}" -f "${ROOT_DIR}/supply-api/sql/postgresql/partition_strategy_v1.sql" + psql_exec "${SUPPLY_DB}" -f "${ROOT_DIR}/supply-api/sql/postgresql/outbox_pattern_v1.sql" + psql_exec "${SUPPLY_DB}" -f "${ROOT_DIR}/supply-api/sql/postgresql/token_status_registry_v1.sql" + psql_exec "${SUPPLY_DB}" -f "${ROOT_DIR}/supply-api/sql/postgresql/audit_alerts_v1.sql" + psql_exec "${SUPPLY_DB}" -f "${ROOT_DIR}/sql/postgresql/iam_schema_v1.sql" +} + +apply_token_runtime_schema() { + psql_exec "${TOKEN_DB}" -c "CREATE EXTENSION IF NOT EXISTS pgcrypto;" + psql_exec "${TOKEN_DB}" -f "${ROOT_DIR}/sql/postgresql/token_runtime_schema_v1.sql" +} + +write_env_file() { + cat >"${ENV_FILE}" </dev/null 2>&1; then + echo "[devtest] ${name} already running (pid=${pid})" + return 0 + fi + rm -f "${pid_file}" + fi + + nohup bash -lc "${command}" >"${log_file}" 2>&1 & + local pid=$! + echo "${pid}" >"${pid_file}" + echo "[devtest] started ${name} (pid=${pid})" +} + +wait_http() { + local name="$1" + local url="$2" + local attempts=0 + until curl -fsS "${url}" >/dev/null 2>&1; do + attempts=$((attempts + 1)) + if [[ "${attempts}" -ge 60 ]]; then + echo "[devtest] ${name} did not become ready: ${url}" >&2 + exit 1 + fi + sleep 1 + done +} + +ensure_pg_container +ensure_database "${SUPPLY_DB}" +ensure_database "${TOKEN_DB}" +apply_supply_schema +apply_token_runtime_schema +write_env_file + +start_process "mock-openai" \ + "cd \"${ROOT_DIR}/supply-api\" && GOCACHE=\"${STATE_DIR}/go-cache/mock-openai\" go run ./cmd/devtestctl mock-openai --addr \"${MOCK_OPENAI_ADDR}\" --models \"${OPENAI_MODELS}\"" +wait_http "mock-openai" "http://${MOCK_OPENAI_ADDR}/healthz" + +start_process "platform-token-runtime" \ + "cd \"${ROOT_DIR}/platform-token-runtime\" && TOKEN_RUNTIME_ADDR=\"${TOKEN_RUNTIME_ADDR}\" TOKEN_RUNTIME_ENV=\"prod\" TOKEN_RUNTIME_DATABASE_URL=\"postgres://${PG_USER}:${PG_PASSWORD}@${PG_HOST}:${PG_PORT}/${TOKEN_DB}?sslmode=disable\" GOCACHE=\"${STATE_DIR}/go-cache/token-runtime\" go run ./cmd/platform-token-runtime" +wait_http "platform-token-runtime" "http://${TOKEN_RUNTIME_ADDR}/actuator/health" + +start_process "supply-api" \ + "cd \"${ROOT_DIR}/supply-api\" && SUPPLY_API_ADDR=\"${SUPPLY_API_ADDR}\" SUPPLY_DB_HOST=\"${PG_HOST}\" SUPPLY_DB_PORT=\"${PG_PORT}\" SUPPLY_DB_USER=\"${PG_USER}\" SUPPLY_DB_PASSWORD=\"${PG_PASSWORD}\" SUPPLY_DB_NAME=\"${SUPPLY_DB}\" SUPPLY_API_IAM_ENABLED=\"true\" SUPPLY_TOKEN_SECRET_KEY=\"${SUPPLY_TOKEN_SECRET_KEY}\" SUPPLY_TOKEN_ISSUER=\"${SUPPLY_TOKEN_ISSUER}\" GOCACHE=\"${STATE_DIR}/go-cache/supply-api\" go run ./cmd/supply-api -env=staging -config ./config/config.dev.yaml" +wait_http "supply-api" "http://127.0.0.1${SUPPLY_API_ADDR#:}/actuator/health" + +start_process "gateway" \ + "cd \"${ROOT_DIR}/gateway\" && GATEWAY_HOST=\"${GATEWAY_HOST}\" GATEWAY_PORT=\"${GATEWAY_PORT}\" GATEWAY_ENV=\"staging\" GATEWAY_TOKEN_RUNTIME_MODE=\"remote_introspection\" GATEWAY_TOKEN_RUNTIME_URL=\"http://${TOKEN_RUNTIME_ADDR}\" OPENAI_BASE_URL=\"http://${MOCK_OPENAI_ADDR}\" OPENAI_API_KEY=\"mock-devtest-key\" OPENAI_MODELS=\"${OPENAI_MODELS}\" GOCACHE=\"${STATE_DIR}/go-cache/gateway\" go run ./cmd/gateway" +wait_http "gateway" "http://${GATEWAY_HOST}:${GATEWAY_PORT}/health" + +echo "[devtest] stack is ready" +echo "[devtest] env file: ${ENV_FILE}" diff --git a/scripts/devtest/stop_dev_stack.sh b/scripts/devtest/stop_dev_stack.sh new file mode 100644 index 00000000..9cf53694 --- /dev/null +++ b/scripts/devtest/stop_dev_stack.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +STATE_DIR="${ROOT_DIR}/.tmp/devtest" +PID_DIR="${STATE_DIR}/pids" +WITH_DB="${1:-}" + +stop_pid_file() { + local pid_file="$1" + if [[ ! -f "${pid_file}" ]]; then + return 0 + fi + + local pid + pid="$(cat "${pid_file}")" + if kill -0 "${pid}" >/dev/null 2>&1; then + kill "${pid}" >/dev/null 2>&1 || true + for _ in {1..20}; do + if ! kill -0 "${pid}" >/dev/null 2>&1; then + break + fi + sleep 1 + done + if kill -0 "${pid}" >/dev/null 2>&1; then + kill -9 "${pid}" >/dev/null 2>&1 || true + fi + fi + + rm -f "${pid_file}" +} + +if [[ -d "${PID_DIR}" ]]; then + stop_pid_file "${PID_DIR}/gateway.pid" + stop_pid_file "${PID_DIR}/supply-api.pid" + stop_pid_file "${PID_DIR}/platform-token-runtime.pid" + stop_pid_file "${PID_DIR}/mock-openai.pid" +fi + +if [[ "${WITH_DB}" == "--with-db" ]]; then + if [[ -f "${STATE_DIR}/env.sh" ]]; then + # shellcheck disable=SC1090 + source "${STATE_DIR}/env.sh" + if command -v podman >/dev/null 2>&1 && podman container exists "${LIJIAOQIAO_DEVTEST_PG_CONTAINER}" >/dev/null 2>&1; then + podman stop "${LIJIAOQIAO_DEVTEST_PG_CONTAINER}" >/dev/null 2>&1 || true + fi + fi +fi + +echo "[devtest] stack stopped"