docs(v2): add batch auto-import spec and tdd plan with resolved open questions
- Add BATCH_AUTO_IMPORT_SPEC.md: 3-stage pipeline (probe/provision/validate), provider_id=host+hash, smoke_model=find-first-usable, pricing=defaults - Add BATCH_AUTO_IMPORT_TDD_PLAN.md: 5-stage implementation plan, 10 tasks - Update EXECUTION_BOARD.md: add v2 section with resolved open questions
This commit is contained in:
280
docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md
Normal file
280
docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md
Normal file
@@ -0,0 +1,280 @@
|
||||
# SPEC: Batch Auto-Import by URL + Key (v2)
|
||||
|
||||
日期:2026-05-21
|
||||
|
||||
## 1. Objective
|
||||
|
||||
让管理员只提供一批 `(base_url, api_key)` 对,就能自动完成:
|
||||
|
||||
1. **上游探测** — 调用 `GET {base_url}/v1/models` 动态获取该 key 支持的模型列表
|
||||
2. **宿主演化** — 将发现的模型与宿主 channel 配置对比,自动扩展 `model_mapping`
|
||||
3. **供应商注册** — 把 URL+key 注册为可控可管的 provider
|
||||
4. **中转闭环验证** — 用该 key 跑一次 `/v1/chat/completions` 确认真实可用
|
||||
|
||||
全程**无需预置 provider manifest**,不依赖 pack,零人工判断。
|
||||
|
||||
## 2. 为什么现在需要这个
|
||||
|
||||
当前 v1 依赖预定义 provider manifest(`packs/openai-cn-pack/providers/*.json`),每个 provider 必须手动写好 `base_url / default_models / smoke_test_model / channel_template`。这带来三个问题:
|
||||
|
||||
- **新 key 无法即插即用**:每次接一个陌生 provider URL,都得先查文档再写 manifest
|
||||
- **模型列表人工维护**:provider 上游升级模型,pack 里不会自动同步
|
||||
- **调试链路长**:假设备注 manifest → 导入 → 发现 channel 缺少模型 → 手动补 → 重新导入
|
||||
|
||||
v2 把"探测 → 配置 → 注册 → 验证"压缩成**一键闭环**。
|
||||
|
||||
## 3. 核心用户故事
|
||||
|
||||
> 作为管理员,我有了一批新的中转 key(URL + token),我想在已经运行的宿主上快速开通这些模型。理想情况是我把这批 key 列出来,系统自动探测每个 key 支持什么模型、自动配置宿主 channel、自动注册为可控 provider、自动跑一遍真实 completion 测试,最后告诉我哪些真正可用。
|
||||
|
||||
## 4. 技术方案
|
||||
|
||||
### 4.1 三阶段管道
|
||||
|
||||
```
|
||||
输入: [(base_url, api_key), ...]
|
||||
|
||||
Stage 1: Probe ─────────────────────────────────────────────────
|
||||
for each (url, key):
|
||||
upstream_models = GET {url}/v1/models
|
||||
→ extract model list
|
||||
upstream_completion = POST {url}/v1/chat/completions (smoke)
|
||||
→ HTTP status, latency, error_type
|
||||
classify: models_ok | models_fail | completion_fail | unreachable
|
||||
|
||||
Stage 2: Provision ──────────────────────────────────────────────
|
||||
for each (url, key) where upstream_models != models_fail:
|
||||
host_channel = find_or_create_channel(provider_id, url)
|
||||
missing_models = upstream_models - host_channel.model_mapping.keys
|
||||
if missing_models:
|
||||
patch_channel(host_channel, add model_mapping entries)
|
||||
managed_account = create_or_update_account(url, key)
|
||||
probe_result = account_test(managed_account, smoke_test_model)
|
||||
register_provider_binding(provider_id, url, key, upstream_models)
|
||||
|
||||
Stage 3: Validate ───────────────────────────────────────────────
|
||||
for each registered (url, key):
|
||||
final_completion = POST host_gw/v1/chat/completions
|
||||
via managed_account key
|
||||
→ write access_status: active | broken | degraded
|
||||
output: per-url status + summary
|
||||
|
||||
输出: BatchImportResult {
|
||||
total: int
|
||||
active: int
|
||||
broken: int
|
||||
degraded: int
|
||||
details: [{url, upstream_models, channel_config, access_status, error}]
|
||||
}
|
||||
```
|
||||
|
||||
### 4.2 关键设计决策
|
||||
|
||||
#### Q1: 如何从 `/v1/models` 提取模型列表?
|
||||
|
||||
OpenAI-compatible 上游返回格式为:
|
||||
```json
|
||||
{
|
||||
"data": [{"id": "gpt-4", "object": "model", ...}, ...]
|
||||
}
|
||||
```
|
||||
|
||||
提取策略:
|
||||
- 取 `data[].id` 作为模型名
|
||||
- 过滤掉以 `gpt-` / `claude-` / `text-` / `embedding-` 开头的明显非目标模型
|
||||
- 保留其余作为"发现的模型列表"
|
||||
|
||||
#### Q2: 如何把上游模型写入宿主 channel?
|
||||
|
||||
宿主 channel 有两个相关字段:
|
||||
- `model_mapping: map[string]string` — `{upstream_model: gateway_model}`
|
||||
- `restrict_models: bool` — true 时 gateway 只路由 mapping 内的模型
|
||||
|
||||
策略:
|
||||
- `model_mapping[key] = key`(一对一映射,上游模型名即 gateway 模型名)
|
||||
- `model_pricing` 填默认值(`price_per_1m=0`, `max_batch=0`),不阻塞导入
|
||||
- 如果 channel 不存在,创建新 channel(`name = host_registered_{provider_id}`)
|
||||
|
||||
#### Q3: Provider ID 如何生成?
|
||||
|
||||
自动生成规则:
|
||||
- 取 `base_url` 的 host 部分,规范化(去掉 `https://`、去除尾部 `/`)
|
||||
- 去除常见后缀(`.com`、`.cn`)
|
||||
- 转小写 + 中划线连接
|
||||
- 示例:`https://api.deepseek.com` → `api-deepseek`
|
||||
|
||||
这样同一 URL 的多次导入会命中同一个 provider_id,实现增量更新。
|
||||
|
||||
#### Q4: 如何避免重复 key 覆盖已有配置?
|
||||
|
||||
导入前执行 reconcile:
|
||||
- 如果 `base_url + key` 对应的 account 已存在,且 `upstream_models` 与已有 account 的 `credentials.model_mapping` 一致 → 跳过
|
||||
- 如果 account 存在但模型列表变长了 → patch channel 扩展 model_mapping
|
||||
- 如果 account 存在但 key 已失效 → 标记为 `broken`,新建 account
|
||||
|
||||
#### Q5: 验证 key 失效 vs 上游断连如何区分?
|
||||
|
||||
Stage 1 的 smoke test 需要区分错误类型:
|
||||
- `401/403 unauthorized` → key 无效
|
||||
- `429 rate_limit` → key 有额度但被限流 → 记录,不阻塞
|
||||
- `502/503/connection_error` → 上游不可达 → 降级处理
|
||||
- `200 + valid response` → key 可用
|
||||
|
||||
Stage 3 的 host relay smoke 测试结果才决定最终 `access_status`。
|
||||
|
||||
### 4.3 数据流
|
||||
|
||||
```
|
||||
BatchImportRequest
|
||||
├── base_url: string
|
||||
├── api_key: string
|
||||
└── access_mode: "subscription" | "self_service" (可选,默认 subscription)
|
||||
|
||||
BatchImportResult
|
||||
├── batch_id: string
|
||||
├── total: int
|
||||
├── active: int
|
||||
├── broken: int
|
||||
├── degraded: int
|
||||
└── results: []ImportItemResult
|
||||
|
||||
ImportItemResult
|
||||
├── base_url: string
|
||||
├── provider_id: string (自动生成)
|
||||
├── upstream_models: []string (Stage 1 发现)
|
||||
├── channel_id: int64 (Stage 2 创建/更新)
|
||||
├── account_id: int64 (Stage 2 创建/更新)
|
||||
├── probe_ok: bool (Stage 2 account test)
|
||||
├── access_status: string (Stage 3 最终)
|
||||
└── error: string | null
|
||||
```
|
||||
|
||||
### 4.4 CLI 接口
|
||||
|
||||
```bash
|
||||
# 单条
|
||||
go run ./cmd/cli batch-import \
|
||||
--host-base-url http://localhost:18097 \
|
||||
--host-api-key <admin-key> \
|
||||
--entry "https://api.deepseek.com,<deepseek-key>" \
|
||||
--access-mode subscription
|
||||
|
||||
# 批量(文件,每行 url,key)
|
||||
go run ./cmd/cli batch-import \
|
||||
--host-base-url http://localhost:18097 \
|
||||
--host-api-key <admin-key> \
|
||||
--batch-file ./keys.csv \
|
||||
--access-mode subscription
|
||||
|
||||
# 批量(stdin)
|
||||
cat keys.txt | xargs -I{} go run ./cmd/cli batch-import \
|
||||
--host-base-url http://localhost:18097 \
|
||||
--host-api-key <admin-key> \
|
||||
--batch-stdin
|
||||
```
|
||||
|
||||
`keys.csv` 格式:
|
||||
```csv
|
||||
https://api.deepseek.com,sk-xxx
|
||||
https://api.completion.com,sk-yyy
|
||||
```
|
||||
|
||||
## 5. 宿主硬约束(继承自 v1)
|
||||
|
||||
- 不修改宿主源码
|
||||
- 不直接写宿主数据库
|
||||
- 只通过宿主 HTTP Admin API 和 Gateway API 工作
|
||||
- channel 完整收口字段必须同时存在:`model_mapping` + `model_pricing` + `restrict_models=true` + `billing_model_source=channel_mapped`
|
||||
- `/v1/models` 和 `/v1/chat/completions` 是两个独立验收层
|
||||
|
||||
## 6. 访问闭环
|
||||
|
||||
Stage 3 的 `access_status` 决定真实可用性:
|
||||
|
||||
| access_status | 含义 | 用户可使用 |
|
||||
|---|---|---|
|
||||
| `active` | Stage1 probe OK + Stage2 account OK + Stage3 completion OK | ✅ |
|
||||
| `degraded` | Stage1/2 OK,但 Stage3 completion 异常 | ⚠️ 限流/不稳定 |
|
||||
| `broken` | Stage1 probe 失败或 Stage2 account test 失败 | ❌ |
|
||||
|
||||
## 7. 错误恢复策略
|
||||
|
||||
- Stage 1 失败:记录 `upstream_unreachable`,跳过 Stage 2/3
|
||||
- Stage 2 部分失败:已完成资源保留(不自动回滚)
|
||||
- Stage 3 失败:access_status 降级,但已创建资源不删除
|
||||
- 整批中断:按 `--mode strict | partial` 处理
|
||||
- `strict`:任一 item 失败,整批停止,报告已完成的
|
||||
- `partial`(默认):失败 item 单独记录,成功的继续
|
||||
|
||||
## 8. 与 v1 的关系
|
||||
|
||||
v2 **不取代** v1,而是新增一条并行入口:
|
||||
|
||||
| | v1 (Pack-Based) | v2 (Auto-Import) |
|
||||
|---|---|---|
|
||||
| 输入 | provider manifest | URL + API key |
|
||||
| 模型来源 | pack 内置 | 上游动态探测 |
|
||||
| 适用场景 | 已知 provider,批量标准化导入 | 新 provider,即插即用 |
|
||||
| channel 配置 | manifest 预定义 | 自动发现 + 扩展 |
|
||||
|
||||
v2 的 provider binding 复用 v1 已有 `managed_resources` 和 `import_batches` 表,只是入口不同。
|
||||
|
||||
## 9. 项目结构变化
|
||||
|
||||
```
|
||||
internal/
|
||||
probe/ # 新增:上游探测模块
|
||||
models.go # GET /v1/models 解析
|
||||
completion.go # smoke test POST /v1/chat/completions
|
||||
classifier.go # 错误分类(auth/rate_limit/upstream/unreachable)
|
||||
batch/ # 新增:批量导入编排
|
||||
service.go # BatchImportService: 管道编排
|
||||
provider_id.go # URL → provider_id 规范化
|
||||
channel_evolution.go # model_mapping 扩展逻辑
|
||||
host/sub2api/
|
||||
channel.go # 新增: PatchChannel(channel_id, add_model_mapping)
|
||||
cmd/
|
||||
cli/
|
||||
batch_import.go # 新增: batch-import 命令
|
||||
tests/integration/
|
||||
batch_import_test.go # 新增: 批量导入集成测试
|
||||
```
|
||||
|
||||
## 10. 测试策略
|
||||
|
||||
### 单测
|
||||
- `probe/models_test.go` — 模型列表解析,覆盖 OpenAI 格式变体
|
||||
- `probe/classifier_test.go` — 错误类型分类
|
||||
- `batch/provider_id_test.go` — URL → provider_id 规范化
|
||||
- `batch/channel_evolution_test.go` — model_mapping 扩展差异计算
|
||||
- `batch/service_test.go` — 管道编排 mock 测试
|
||||
|
||||
### 集成测
|
||||
- `tests/integration/batch_import_test.go`
|
||||
- 两组 (url, key),probe + provision + validate 全流程
|
||||
- strict 模式任一失败整批停止
|
||||
- partial 模式失败 item 隔离
|
||||
|
||||
## 11. 暂不做(v2 范围外)
|
||||
|
||||
- Web UI / HTTP API 入口(CLI 先跑通)
|
||||
- 自动发现 provider 的 channel pricing(model pricing 留空,等用户配置)
|
||||
- 多 key 之间的负载均衡策略
|
||||
- 对账调度器( reconcile 由 v1 提供)
|
||||
|
||||
## 12. 成功标准
|
||||
|
||||
1. CLI `batch-import` 可接受单条和文件批量输入
|
||||
2. Stage 1 probe 能在 10s 内返回上游模型列表(超时控制)
|
||||
3. 重复导入同一 URL+key 时,不重复创建 channel/account(幂等)
|
||||
4. Stage 3 completion 测试通过时,`access_status=active`
|
||||
5. Stage 3 失败时,access_status 正确降级(broken/degraded)
|
||||
6. `strict` 模式下,任一 item 失败整批停止并报告
|
||||
7. `partial` 模式下,成功的 item 不因失败 item 而中断
|
||||
8. 全流程不修改宿主源码,不写宿主数据库
|
||||
|
||||
## 13. 开放问题(已决策)
|
||||
|
||||
1. **provider_id 策略**:选 B(host + hash),`{normalized_host}-{url_hash_last8}`
|
||||
2. **model_pricing 为空**:选 B,自动补空 pricing(填默认值,不阻塞导入)
|
||||
3. **smoke test model**:选 C,遍历 data 找第一个能完成 chat completion 的模型
|
||||
269
docs/2026-05-21-BATCH_AUTO_IMPORT_TDD_PLAN.md
Normal file
269
docs/2026-05-21-BATCH_AUTO_IMPORT_TDD_PLAN.md
Normal file
@@ -0,0 +1,269 @@
|
||||
# TDD 实施计划 v2 — Batch Auto-Import
|
||||
|
||||
日期:2026-05-21
|
||||
|
||||
## 依赖顺序
|
||||
|
||||
必须按以下顺序实现,前一个未完成前不开始后一个:
|
||||
|
||||
```
|
||||
probe/models → probe/classifier
|
||||
↓ ↓
|
||||
└──────→ batch/service ←── host/channel_patch
|
||||
↓
|
||||
cmd/cli/batch_import
|
||||
↓
|
||||
tests/integration/batch_import
|
||||
```
|
||||
|
||||
## Stage 1: probe 模块(上游探测)
|
||||
|
||||
### 1.1 `internal/probe/models.go`
|
||||
|
||||
**职责**:调用 `GET {base_url}/v1/models`,解析 OpenAI 格式响应。
|
||||
|
||||
```go
|
||||
// ProviderModels returns the list of model IDs from a provider's /v1/models endpoint.
|
||||
func ProviderModels(ctx context.Context, baseURL, apiKey string) ([]string, error)
|
||||
|
||||
// Classifier errors into:
|
||||
// - ErrAuthFailed : 401/403
|
||||
// - ErrRateLimited : 429
|
||||
// - ErrUpstreamUnreachable : 502/503/timeout/connection
|
||||
// - ErrUnexpected : 其他 HTTP 错误
|
||||
```
|
||||
|
||||
**单测**:
|
||||
```go
|
||||
func TestProviderModels_OpenAIFormat_ReturnsModelList(t *testing.T)
|
||||
func TestProviderModels_FilterOutNonChatModels(t *testing.T)
|
||||
func TestProviderModels_EmptyData_ReturnsEmptySlice(t *testing.T)
|
||||
func TestProviderModels_AuthFailed_ReturnsErrAuthFailed(t *testing.T)
|
||||
func TestProviderModels_Timeout_ReturnsErrUpstreamUnreachable(t *testing.T)
|
||||
```
|
||||
|
||||
### 1.2 `internal/probe/classifier.go`
|
||||
|
||||
**职责**:对 HTTP 响应/错误进行分类,返回结构化 ProbeResult。
|
||||
|
||||
```go
|
||||
type ProbeResult struct {
|
||||
URL string
|
||||
HTTPStatus int
|
||||
Models []string
|
||||
Classification string // "auth_failed" | "rate_limited" | "unreachable" | "ok"
|
||||
LatencyMs int64
|
||||
Error string
|
||||
}
|
||||
```
|
||||
|
||||
**单测**:
|
||||
```go
|
||||
func TestClassify_401_ReturnsAuthFailed(t *testing.T)
|
||||
func TestClassify_429_ReturnsRateLimited(t *testing.T)
|
||||
func TestClassify_502_ReturnsUpstreamUnreachable(t *testing.T)
|
||||
func TestClassify_200_ReturnsOk(t *testing.T)
|
||||
```
|
||||
|
||||
### 1.3 `internal/probe/completion.go`
|
||||
|
||||
**职责**:遍历 `/v1/models` 返回的 data,找第一个能完成 chat completion 的模型并执行 smoke test。
|
||||
|
||||
```go
|
||||
// FindSmokeModel traverses the model list and returns the first model
|
||||
// that successfully completes a chat completion request.
|
||||
func FindSmokeModel(ctx context.Context, baseURL, apiKey string, models []string) (model string, result *CompletionResult, err error)
|
||||
|
||||
// DefaultModelPricing returns a minimal pricing entry for a model
|
||||
// (used when upstream has no pricing data).
|
||||
type DefaultModelPricing struct {
|
||||
Model string
|
||||
PricePer1M float64 // default: 0 (unset)
|
||||
MaxBatch int // default: 0 (unset)
|
||||
}
|
||||
```
|
||||
|
||||
**单测**:
|
||||
```go
|
||||
func TestFindSmokeModel_FirstModelSucceeds_ReturnsIt(t *testing.T)
|
||||
func TestFindSmokeModel_FirstFailsSecondSucceeds_SkipsFirst(t *testing.T)
|
||||
func TestFindSmokeModel_AllFail_ReturnsErrNoUsableModel(t *testing.T)
|
||||
func TestFindSmokeModel_TimeoutBudget_StopsAfterLimit(t *testing.T)
|
||||
func TestDefaultModelPricing_ReturnsZeroValues(t *testing.T)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stage 2: batch 模块(批量导入编排)
|
||||
|
||||
### 2.1 `internal/batch/provider_id.go`
|
||||
|
||||
**决策**:选 B,完整 URL 作为 provider_id 一部分(`{normalized_host}-{url_hash_last8}`)。
|
||||
|
||||
```go
|
||||
// NormalizeProviderID converts a base URL into a stable provider ID using host + hash.
|
||||
// https://api.deepseek.com/v1 → api-deepseek-<last8-of-url-hash>
|
||||
// Collision-resistant: same full URL always produces the same ID.
|
||||
func NormalizeProviderID(baseURL string) string {
|
||||
u, _ := url.Parse(baseURL)
|
||||
host := strings.ToLower(strings.ReplaceAll(u.Host, ":", "-"))
|
||||
hash := fmt.Sprintf("%x", md5.Sum([]byte(baseURL)))[:8]
|
||||
return host + "-" + hash
|
||||
}
|
||||
```
|
||||
|
||||
**单测**:
|
||||
```go
|
||||
func TestNormalizeProviderID_Basic(t *testing.T)
|
||||
func TestNormalizeProviderID_WithPath_IncludesPathHash(t *testing.T)
|
||||
func TestNormalizeProviderID_Idempotent(t *testing.T)
|
||||
func TestNormalizeProviderID_DifferentPaths_DifferentIDs(t *testing.T) // v1 vs v2 不同 hash
|
||||
func TestNormalizeProviderID_SanitizesPort(t *testing.T)
|
||||
```
|
||||
|
||||
### 2.2 `internal/batch/channel_evolution.go`
|
||||
|
||||
**职责**:计算 channel 现有 model_mapping 与新探测模型的差异,返回需要 patch 的内容。
|
||||
|
||||
```go
|
||||
// ModelMappingDelta computes which models need to be added to an existing channel.
|
||||
func ModelMappingDelta(existing []string, discovered []string) (add []string)
|
||||
|
||||
// BuildPatchModelMapping returns the full patched model_mapping for a channel.
|
||||
func BuildPatchModelMapping(existing models map[string]string, add []string) map[string]string
|
||||
```
|
||||
|
||||
**单测**:
|
||||
```go
|
||||
func TestModelMappingDelta_NoOverlap_AddsAll(t *testing.T)
|
||||
func TestModelMappingDelta_FullOverlap_ReturnsEmpty(t *testing.T)
|
||||
func TestModelMappingDelta_PartialOverlap_AddsMissingOnly(t *testing.T)
|
||||
func TestBuildPatchModelMapping_AddsWithIdentityMapping(t *testing.T)
|
||||
```
|
||||
|
||||
### 2.3 `internal/batch/service.go`
|
||||
|
||||
**职责**:编排 Stage 1 + 2 + 3 管道,调用 probe + provision + access。
|
||||
|
||||
```go
|
||||
type BatchImportService struct {
|
||||
host hostadapter.HostAdapter
|
||||
probe *probe.Client
|
||||
provision *provision.ImportService
|
||||
}
|
||||
|
||||
func (s *BatchImportService) ImportBatch(ctx context.Context, req BatchImportRequest) (*BatchImportResult, error)
|
||||
```
|
||||
|
||||
**单测**(mock 外部 HTTP):
|
||||
```go
|
||||
func TestBatchImport_AllProbeOk_ProvisionsAndValidates(t *testing.T)
|
||||
func TestBatchImport_ProbeFails_SkipsProvision(t *testing.T)
|
||||
func TestBatchImport_CompletionFail_ReportsBroken(t *testing.T)
|
||||
func TestBatchImport_StrictMode_StopsOnFirstFailure(t *testing.T)
|
||||
func TestBatchImport_PartialMode_ContinuesOnFailure(t *testing.T)
|
||||
func TestBatchImport_Idempotent_SkipsExistingAccount(t *testing.T)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stage 3: host adapter 扩展
|
||||
|
||||
### 3.1 `internal/host/sub2api/channel.go`
|
||||
|
||||
新增:
|
||||
```go
|
||||
// PatchChannel extends an existing channel's model_mapping with additional models.
|
||||
func (h *HostAdapter) PatchChannel(ctx context.Context, channelID int64, addModels []string) error
|
||||
```
|
||||
|
||||
**单测**(httptest):
|
||||
```go
|
||||
func TestPatchChannel_AddsModelMappingEntries(t *testing.T)
|
||||
func TestPatchChannel_ChannelNotFound_ReturnsError(t *testing.T)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stage 4: CLI
|
||||
|
||||
### 4.1 `cmd/cli/batch_import.go`
|
||||
|
||||
```bash
|
||||
go run ./cmd/cli batch-import \
|
||||
--host-base-url string (required)
|
||||
--host-api-key string (required)
|
||||
--entry "url,key" (单条,与 --batch-file 互斥)
|
||||
--batch-file string (批量文件路径)
|
||||
--mode "strict" | "partial" (default: partial)
|
||||
--access-mode "subscription" | "self_service" (default: subscription)
|
||||
```
|
||||
|
||||
**文件格式**:
|
||||
- `--batch-file`:CSV,每行 `base_url,api_key`(逗号分隔,空行忽略,`#` 开头为注释)
|
||||
|
||||
**输出格式**:
|
||||
```json
|
||||
{
|
||||
"batch_id": "batch-20260521-001",
|
||||
"total": 3,
|
||||
"active": 2,
|
||||
"broken": 1,
|
||||
"degraded": 0,
|
||||
"results": [
|
||||
{"url": "https://api.deepseek.com", "provider_id": "api-deepseek",
|
||||
"upstream_models": ["deepseek-chat", "deepseek-reasoner"],
|
||||
"channel_id": 10, "account_id": 20,
|
||||
"probe_ok": true, "access_status": "active", "error": null},
|
||||
{"url": "https://api.fail.com", "provider_id": "api-fail",
|
||||
"upstream_models": [], "probe_ok": false,
|
||||
"access_status": "broken", "error": "upstream_unreachable"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stage 5: 集成测试
|
||||
|
||||
### `tests/integration/batch_import_test.go`
|
||||
|
||||
使用真实 httptest server 模拟上游 provider:
|
||||
```go
|
||||
func TestBatchImport_FullPipeline(t *testing.T)
|
||||
func TestBatchImport_StrictStopsOnFailure(t *testing.T)
|
||||
func TestBatchImport_PartialContinuesOnFailure(t *testing.T)
|
||||
func TestBatchImport_IdempotentOnDuplicateURLKey(t *testing.T)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 验收命令
|
||||
|
||||
```bash
|
||||
go test ./internal/probe/... -v -count=1
|
||||
go test ./internal/batch/... -v -count=1
|
||||
go test ./internal/host/sub2api/... -v -count=1 -run TestPatchChannel
|
||||
go test ./tests/integration/batch_import_test.go -v -count=1
|
||||
go vet ./...
|
||||
gofmt -l .
|
||||
```
|
||||
|
||||
覆盖率目标:
|
||||
- `internal/probe`: >= 80%
|
||||
- `internal/batch`: >= 75%
|
||||
|
||||
---
|
||||
|
||||
## 任务清单
|
||||
|
||||
- [ ] `internal/probe/models.go` + models_test.go
|
||||
- [ ] `internal/probe/classifier.go` + classifier_test.go
|
||||
- [ ] `internal/probe/completion.go` + completion_test.go
|
||||
- [ ] `internal/batch/provider_id.go` + provider_id_test.go
|
||||
- [ ] `internal/batch/channel_evolution.go` + channel_evolution_test.go
|
||||
- [ ] `internal/host/sub2api/channel.go` PatchChannel + test
|
||||
- [ ] `internal/batch/service.go` + service_test.go
|
||||
- [ ] `cmd/cli/batch_import.go`
|
||||
- [ ] `tests/integration/batch_import_test.go`
|
||||
- [ ] 全量门禁(gofmt / vet / test / race / cover)
|
||||
@@ -185,9 +185,55 @@
|
||||
3. 若换 key 后 upstream `/chat/completions` 变成 `200`,再看 host `/chat/completions` 是否仍有兼容性问题
|
||||
4. 当前代码状态可维持 “代码侧 `CONDITIONAL_APPROVED`、外部门禁 `BLOCKED`”
|
||||
|
||||
## v2 规划:Batch Auto-Import(URL + Key)
|
||||
|
||||
**文档**:`docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md`(需求规格)
|
||||
**TDD 计划**:`docs/2026-05-21-BATCH_AUTO_IMPORT_TDD_PLAN.md`(实现路径)
|
||||
|
||||
**核心目标**:管理员只需提供 `(base_url, api_key)` 列表,系统自动完成上游模型探测 → 宿主 channel 扩展 → provider 注册 → 中转闭环验证,全程无需预置 provider manifest。
|
||||
|
||||
**三阶段管道**:
|
||||
```
|
||||
Stage 1 (Probe) → GET /v1/models 获取模型列表,smoke test completion
|
||||
Stage 2 (Provision) → channel 自动扩展 model_mapping,创建/更新 account
|
||||
Stage 3 (Validate) → host relay completion 测试,写 access_status
|
||||
```
|
||||
|
||||
**新增模块**:
|
||||
- `internal/probe/` — 上游探测(models 解析 / completion smoke / 错误分类)
|
||||
- `internal/batch/` — 批量编排(URL→provider_id / model_mapping 扩展 / 管道服务)
|
||||
- `cmd/cli/batch_import.go` — CLI 入口
|
||||
- `internal/host/sub2api/channel.go` — PatchChannel 扩展
|
||||
|
||||
**成功标准**:
|
||||
- 单条和文件批量输入均可
|
||||
- 重复导入同一 URL+key 幂等(不重复创建 channel/account)
|
||||
- strict/partial 两种模式
|
||||
- Stage 3 completion 通过时 `access_status=active`,失败时正确降级
|
||||
- 全流程不修改宿主源码,不写宿主数据库
|
||||
|
||||
**任务清单**(共 10 项,详见 TDD_PLAN):
|
||||
- [ ] probe/models + test
|
||||
- [ ] probe/classifier + test
|
||||
- [ ] probe/completion + test
|
||||
- [ ] batch/provider_id + test
|
||||
- [ ] batch/channel_evolution + test
|
||||
- [ ] host/channel.go PatchChannel + test
|
||||
- [ ] batch/service + test
|
||||
- [ ] cmd/cli/batch_import
|
||||
- [ ] integration test
|
||||
- [ ] 全量门禁
|
||||
|
||||
**开放问题(已决策)**:
|
||||
1. ~~smoke test model 选择策略~~ → 选 C:遍历找第一个可用的
|
||||
2. ~~model_pricing 为空时 restrict_models 行为~~ → 选 B:自动补空 pricing
|
||||
3. ~~provider_id 冲突策略~~ → 选 B:host + hash(`{normalized_host}-{url_hash_last8}`)
|
||||
|
||||
---
|
||||
|
||||
## 禁止错误结论
|
||||
|
||||
- ❌ 历史失败 artifact ≠ 当前 fresh redeploy 仍失败
|
||||
- ❌ capability probe 无副作用 ≠ 所有宿主版本都已真实兼容
|
||||
- ❌ rollback-provider 已改安全路径 ≠ 历史脏资源自动消失
|
||||
|- ❌ 历史失败 artifact ≠ 当前 fresh redeploy 仍失败
|
||||
|- ❌ capability probe 无副作用 ≠ 所有宿主版本都已真实兼容
|
||||
|- ❌ rollback-provider 已改安全路径 ≠ 历史脏资源自动消失
|
||||
- ❌ `HTTP 200` ≠ 宿主初始化会自动准备普通用户/订阅/余额;这些仍是显式运营前置
|
||||
|
||||
Reference in New Issue
Block a user