feat: expand AI workflow support and refresh docs
This commit is contained in:
40
Server/.env.example
Normal file
40
Server/.env.example
Normal file
@@ -0,0 +1,40 @@
|
||||
ENV=development
|
||||
PROJECT_NAME=DesignerCEP
|
||||
API_V1_STR=/api/v1
|
||||
DATABASE_URL=sqlite:///./designercep.db
|
||||
SECRET_KEY=replace-with-a-long-random-secret
|
||||
ACCESS_TOKEN_EXPIRE_MINUTES=10080
|
||||
ALLOWED_ORIGINS=*
|
||||
ADMIN_TOKEN=replace-with-admin-token
|
||||
|
||||
# SMTP
|
||||
SMTP_HOST=smtp.gmail.com
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=
|
||||
SMTP_PASSWORD=
|
||||
EMAILS_FROM_EMAIL=
|
||||
EMAILS_FROM_NAME=Designer
|
||||
|
||||
# 七牛云对象存储
|
||||
QINIU_ACCESS_KEY=
|
||||
QINIU_SECRET_KEY=
|
||||
QINIU_BUCKET=
|
||||
QINIU_DOMAIN=http://image.xinhui.cloud
|
||||
|
||||
# AI（推荐统一使用 AI_*）
|
||||
AI_PROVIDER=ark
|
||||
AI_API_KEY=
|
||||
AI_BASE_URL=https://ark.cn-beijing.volces.com/api/v3
|
||||
AI_CHAT_BASE_URL=
|
||||
AI_VISION_BASE_URL=
|
||||
AI_IMAGE_BASE_URL=
|
||||
AI_MODEL=doubao-seed-2-0-mini-260215
|
||||
AI_VISION_MODEL=doubao-seed-2-0-mini-260215
|
||||
AI_IMAGE_EDIT_MODEL=doubao-seedream-5-0-260128
|
||||
AI_CHAT_MODELS=doubao-seed-2-0-mini-260215
|
||||
AI_VISION_MODELS=doubao-seed-2-0-mini-260215
|
||||
AI_IMAGE_EDIT_MODELS=doubao-seedream-5-0-260128
|
||||
|
||||
# 兼容旧字段
|
||||
ARK_API_KEY=
|
||||
ARK_BASE_URL=https://ark.cn-beijing.volces.com/api/v3
|
||||
164
Server/README.md
164
Server/README.md
@@ -1,109 +1,103 @@
|
||||
# 📁 Server 目录说明
|
||||
# Server（FastAPI 后端）
|
||||
|
||||
## 目录结构
|
||||
当前目录是 DesignerCEP 的主后端服务。
|
||||
|
||||
```
|
||||
## 1. 当前目录结构（关键）
|
||||
|
||||
```text
|
||||
Server/
|
||||
├── app/ ← 后端代码（FastAPI）
|
||||
│ ├── api/ ← API 路由
|
||||
│ ├── core/ ← 核心配置
|
||||
│ ├── models/ ← 数据库模型
|
||||
│ ├── schemas/ ← 数据验证
|
||||
│ ├── services/ ← 业务逻辑
|
||||
│ ├── main.py ← 后端入口
|
||||
│ └── db.py ← 数据库连接
|
||||
│
|
||||
├── static/ ← ⭐ 前端静态文件（重要！）
|
||||
│ └── app/ ← 前端构建产物放这里
|
||||
│ ├── index.html ← 前端入口
|
||||
│ ├── assets/ ← JS/CSS/图片
|
||||
│ └── ...
|
||||
│
|
||||
├── archives/ ← Core 版本压缩包（历史版本）
|
||||
│ ├── core-v1.0.0.zip
|
||||
│ ├── core-v1.0.1.zip
|
||||
│ └── ...
|
||||
│
|
||||
├── tests/ ← 测试代码
|
||||
├── routers/ ← 额外的路由（如果有）
|
||||
│
|
||||
├── docker-compose.yml ← Docker 编排配置
|
||||
├── Dockerfile ← 后端镜像构建
|
||||
├── requirements.txt ← Python 依赖
|
||||
├── mysql.cnf ← MySQL 配置
|
||||
├── .dockerignore ← Docker 忽略文件
|
||||
│
|
||||
└── designercep.db ← SQLite 数据库（本地开发用）
|
||||
├── app/ # FastAPI 代码
|
||||
│ ├── api/v1/ # 接口路由
|
||||
│ ├── core/ # 配置与安全
|
||||
│ ├── models/ # ORM 模型
|
||||
│ ├── schemas/ # Pydantic Schema
|
||||
│ ├── services/ # 业务服务
|
||||
│ ├── db.py # SQLAlchemy 初始化与 seed
|
||||
│ └── main.py # 应用入口
|
||||
├── archives/ # 上传版本包归档目录
|
||||
├── debug_images/ # 调试图片输出
|
||||
├── docker-compose.yml # Docker 编排（backend + mysql + pma + portainer）
|
||||
├── Dockerfile # 后端镜像
|
||||
├── requirements.txt # Python 依赖
|
||||
├── start.sh # 容器启动脚本
|
||||
├── .env.example # 配置模板（推荐复制后本地填写）
|
||||
└── .env # 运行配置（敏感）
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⭐ 前端文件应该放在这里
|
||||
|
||||
### 构建前端后，复制到这里：
|
||||
|
||||
```
|
||||
Server/static/app/
|
||||
├── index.html ← 前端入口（重要！）
|
||||
├── assets/ ← JS/CSS 等资源
|
||||
│ ├── index-xxx.js
|
||||
│ ├── index-xxx.css
|
||||
│ └── ...
|
||||
├── CSInterface.js ← CEP 桥接文件
|
||||
└── vite.svg ← 图标等
|
||||
```
|
||||
|
||||
### 操作步骤：
|
||||
|
||||
```bash
|
||||
# 1. 构建前端（在 Designer 目录）
|
||||
cd Designer
|
||||
npm run build:core
|
||||
|
||||
# 2. 复制到 Server/static/app/
|
||||
# Windows:
|
||||
xcopy /E /Y dist_core\* ..\Server\static\app\
|
||||
|
||||
# Linux/Mac:
|
||||
cp -r dist_core/* ../Server/static/app/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🐳 Docker 部署
|
||||
|
||||
### 启动服务
|
||||
## 2. 本地运行
|
||||
|
||||
```bash
|
||||
cd Server
|
||||
docker-compose up -d
|
||||
copy .env.example .env
|
||||
pip install -r requirements.txt
|
||||
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
```
|
||||
|
||||
### 查看日志
|
||||
检查服务：
|
||||
|
||||
```bash
|
||||
docker-compose logs -f
|
||||
curl http://localhost:8000/health
|
||||
```
|
||||
|
||||
### 停止服务
|
||||
## 3. Docker 运行
|
||||
|
||||
```bash
|
||||
cd Server
|
||||
docker-compose up -d --build
|
||||
docker-compose ps
|
||||
docker-compose logs -f backend
|
||||
```
|
||||
|
||||
停止服务：
|
||||
|
||||
```bash
|
||||
docker-compose down
|
||||
```
|
||||
|
||||
---
|
||||
## 4. 配置说明（来自 `app/core/config.py`）
|
||||
|
||||
## 📋 部署清单
|
||||
主要环境变量（优先使用 `AI_*`）：
|
||||
|
||||
- [ ] 前端文件已复制到 `static/app/`
|
||||
- [ ] `.env` 文件已配置
|
||||
- [ ] MySQL 密码已修改
|
||||
- [ ] SECRET_KEY 已修改
|
||||
- [ ] ALLOWED_ORIGINS 已配置
|
||||
- [ ] Docker 服务已启动
|
||||
- [ ] 访问 https://app.aidg168.uk/ 测试
|
||||
- `ENV`（development / production）
|
||||
- `DATABASE_URL`
|
||||
- `SECRET_KEY`
|
||||
- `ADMIN_TOKEN`
|
||||
- `ALLOWED_ORIGINS`
|
||||
- `SMTP_*`
|
||||
- `QINIU_*`
|
||||
- `AI_API_KEY`、`AI_BASE_URL`、`AI_MODEL`
|
||||
- `AI_VISION_MODEL`、`AI_IMAGE_EDIT_MODEL`
|
||||
- `QINIU_DOMAIN`
|
||||
|
||||
---
|
||||
## 5. 数据库迁移
|
||||
|
||||
**重要**：前端文件必须放在 `Server/static/app/` 目录！
|
||||
推荐使用 Alembic：
|
||||
|
||||
```bash
|
||||
cd Server
|
||||
.venv\Scripts\python -m alembic upgrade head
|
||||
```
|
||||
|
||||
新增字段后生成迁移：
|
||||
|
||||
```bash
|
||||
.venv\Scripts\python -m alembic revision --autogenerate -m "describe_change"
|
||||
```
|
||||
|
||||
## 6. 接口前缀与入口
|
||||
|
||||
- API 前缀：`/api/v1`
|
||||
- 健康检查：`GET /health`
|
||||
- 启动入口：`app/main.py`
|
||||
|
||||
## 7. 关于静态文件
|
||||
|
||||
当前仓库中没有 `Server/static/app/` 目录。
|
||||
在线部署场景下，前端静态资源由 Caddy/Nginx 单独托管（仓库根目录 `Caddyfile` 示例为 `/var/www/app`）。
|
||||
|
||||
## 8. 安全注意事项
|
||||
|
||||
- `Server/.env` 含真实密钥，不应继续明文共享。
|
||||
- 启动时如果检测到默认密钥，后端会输出配置警告。
|
||||
- 部署前请至少轮换：`SECRET_KEY`、`ADMIN_TOKEN`、数据库密码、AI 与云存储密钥。
|
||||
|
||||
|
||||
@@ -1,20 +1,21 @@
|
||||
import os
|
||||
import sys
|
||||
from logging.config import fileConfig
|
||||
from pathlib import Path
|
||||
|
||||
from sqlalchemy import engine_from_config
|
||||
from sqlalchemy import pool
|
||||
|
||||
from alembic import context
|
||||
|
||||
# 将当前目录添加到 sys.path，以便能够导入 app 模块
|
||||
sys.path.append(os.getcwd())
|
||||
SERVER_DIR = Path(__file__).resolve().parents[1]
|
||||
if str(SERVER_DIR) not in sys.path:
|
||||
sys.path.append(str(SERVER_DIR))
|
||||
|
||||
# 导入配置和模型
|
||||
from app.core.config import settings
|
||||
from app.db import Base
|
||||
# 导入所有模型以确保它们被注册到 Base.metadata
|
||||
from app.models import user, group, business, session
|
||||
from app.models import business, chat, group, logs, session, user
|
||||
|
||||
# this is the Alembic Config object, which provides
|
||||
# access to the values within the .ini file in use.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
245
Server/app/api/v1/ai_identify.py
Normal file
245
Server/app/api/v1/ai_identify.py
Normal file
@@ -0,0 +1,245 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
AI 裁片识别接口
|
||||
功能：识别裁片部位、分析颜色花样、验证套图结果
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Depends
|
||||
from pydantic import BaseModel
|
||||
from typing import Optional
|
||||
import json, logging
|
||||
from app.core.config import settings
|
||||
from app.core.security import get_current_user
|
||||
from app.db import get_db
|
||||
from sqlalchemy.orm import Session
|
||||
from app.models.user import User
|
||||
from app.api.v1.ai_llm import (
|
||||
call_vision_messages,
|
||||
get_runtime_ai_config,
|
||||
verify_pattern_result,
|
||||
has_ai_config,
|
||||
)
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
if not log.handlers:
|
||||
log.setLevel(logging.INFO)
|
||||
log.propagate = False
|
||||
_h = logging.StreamHandler()
|
||||
_h.setFormatter(
|
||||
logging.Formatter("[%(asctime)s][%(name)s][%(levelname)s] %(message)s")
|
||||
)
|
||||
log.addHandler(_h)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# ==================== 数据模型 ====================
|
||||
|
||||
|
||||
class IdentifyPiecesRequest(BaseModel):
|
||||
canvas_base64: str
|
||||
garment_base64: Optional[str] = None
|
||||
layers_info: str
|
||||
vision_model: Optional[str] = None
|
||||
|
||||
|
||||
class VerifyResultRequest(BaseModel):
|
||||
garment_base64: str
|
||||
canvas_base64: str
|
||||
prompt: Optional[str] = None
|
||||
vision_model: Optional[str] = None
|
||||
|
||||
|
||||
# ==================== 裁片识别接口 ====================
|
||||
|
||||
|
||||
@router.post("/ai/identify-pieces")
|
||||
async def identify_pieces(
|
||||
data: IdentifyPiecesRequest,
|
||||
current_username: str = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""用视觉模型识别裁片部位 + 分析成衣各部位的颜色花样"""
|
||||
user = db.query(User).filter(User.username == current_username).first()
|
||||
runtime_ai_config = get_runtime_ai_config(user)
|
||||
effective_vision_model = (
|
||||
data.vision_model
|
||||
or (user.ai_vision_model if user else None)
|
||||
or settings.AI_VISION_MODEL
|
||||
)
|
||||
if not has_ai_config(runtime_ai_config):
|
||||
raise HTTPException(400, "请先在 API 配置页填写转发 Key，或联系管理员配置默认 AI_API_KEY")
|
||||
|
||||
try:
|
||||
result = _identify_garment_pieces(
|
||||
data.canvas_base64,
|
||||
data.layers_info,
|
||||
data.garment_base64,
|
||||
vision_model=effective_vision_model,
|
||||
runtime_ai_config=runtime_ai_config,
|
||||
)
|
||||
return {"code": 200, "data": result}
|
||||
except Exception as e:
|
||||
log.error(f"裁片识别失败: {e}", exc_info=True)
|
||||
raise HTTPException(500, f"裁片识别失败: {str(e)}")
|
||||
|
||||
|
||||
@router.post("/ai/verify-result")
|
||||
async def verify_result(
|
||||
data: VerifyResultRequest,
|
||||
current_username: str = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""对比原始成衣和套图结果，给出验证反馈"""
|
||||
user = db.query(User).filter(User.username == current_username).first()
|
||||
runtime_ai_config = get_runtime_ai_config(user)
|
||||
effective_vision_model = (
|
||||
data.vision_model
|
||||
or (user.ai_vision_model if user else None)
|
||||
or settings.AI_VISION_MODEL
|
||||
)
|
||||
if not has_ai_config(runtime_ai_config):
|
||||
raise HTTPException(400, "请先在 API 配置页填写转发 Key，或联系管理员配置默认 AI_API_KEY")
|
||||
|
||||
try:
|
||||
feedback = verify_pattern_result(
|
||||
data.garment_base64,
|
||||
data.canvas_base64,
|
||||
data.prompt,
|
||||
vision_model=effective_vision_model,
|
||||
runtime_config=runtime_ai_config,
|
||||
)
|
||||
return {"code": 200, "data": {"feedback": feedback}}
|
||||
except Exception as e:
|
||||
log.error(f"验证失败: {e}", exc_info=True)
|
||||
raise HTTPException(500, f"验证失败: {str(e)}")
|
||||
|
||||
|
||||
# ==================== 核心识别逻辑 ====================
|
||||
|
||||
|
||||
def _identify_garment_pieces(
|
||||
canvas_b64: str,
|
||||
layers_info: str,
|
||||
garment_b64: str = None,
|
||||
vision_model: str = None,
|
||||
runtime_ai_config=None,
|
||||
) -> dict:
|
||||
"""用视觉模型分析画布+成衣，识别裁片部位并判断颜色/花样"""
|
||||
use_model = vision_model or settings.AI_VISION_MODEL
|
||||
|
||||
log.info(f"{'=' * 60}")
|
||||
log.info(f"[IdentifyPieces] 调用视觉模型识别裁片 + 分析花样")
|
||||
log.info(f"[IdentifyPieces] 模型: {use_model}")
|
||||
log.info(
|
||||
f"[IdentifyPieces] 画布: {len(canvas_b64) // 1024}KB, 成衣: {len(garment_b64) // 1024 if garment_b64 else 0}KB"
|
||||
)
|
||||
log.info(f"[IdentifyPieces] 图层:\n{layers_info[:500]}")
|
||||
|
||||
layer_names = []
|
||||
for line in layers_info.strip().split("\n"):
|
||||
name = line.split("(")[0].strip()
|
||||
if name:
|
||||
layer_names.append(name)
|
||||
|
||||
ex1 = layer_names[0] if len(layer_names) > 0 else "图层名1"
|
||||
ex2 = layer_names[1] if len(layer_names) > 1 else "图层名2"
|
||||
ex3 = layer_names[2] if len(layer_names) > 2 else "图层名3"
|
||||
|
||||
if garment_b64:
|
||||
prompt = f"""你需要完成三个任务：
|
||||
|
||||
**任务1：识别裁片部位**
|
||||
Image 2 是 PS 文档中的裁片轮廓图。以下是图层信息：
|
||||
{layers_info}
|
||||
|
||||
请根据形状、大小识别每个**图层组**是什么服装部位（前片、后片、袖子、领口等）。
|
||||
⚠️ mapping 的 key 必须是上面图层信息中的**实际图层组名称**（如 "{ex1}"、"{ex2}" 等），不要自己编名字！
|
||||
|
||||
**任务2：判断花型覆盖模式 pattern_mode**
|
||||
Image 1 是成衣照片。请判断花型属于哪种模式：
|
||||
- "all_over"：所有裁片来自同一块面料（碎花/格子/渐变等整体花型覆盖全身）
|
||||
- "placement"：各裁片独立处理（有的印花有的纯色，如卡通卫衣）
|
||||
- "color_block"：全部纯色拼接
|
||||
|
||||
如果是 all_over，还需提供 fabric_info：
|
||||
- fabric_type: 面料印花类型
|
||||
- pattern_direction: 花型方向
|
||||
- pattern_elements: 花型元素描述
|
||||
- color_transition: 色彩过渡描述
|
||||
- density: 花型密度描述
|
||||
|
||||
**任务3：分析每个裁片**
|
||||
- type: solid / fill_pattern / theme_pattern / mixed_pattern / all_over
|
||||
- solid 和 theme_pattern 必须给 color（hex值）
|
||||
- 如果某个裁片在照片中**看不到**（如后片），设置 visible: false 并推理
|
||||
- 左右对称片标记 symmetric_pairs
|
||||
|
||||
用 JSON 回答（key 用实际图层名）：
|
||||
```json
|
||||
{{
|
||||
"pattern_mode": "all_over",
|
||||
"fabric_info": {{
|
||||
"fabric_type": "数码印花",
|
||||
"pattern_direction": "上下渐变",
|
||||
"pattern_elements": "白色百合花朵 + 灰白格纹底",
|
||||
"color_transition": "从黑色渐变到灰白色",
|
||||
"density": "上部花朵密集大朵，下部稀疏小朵"
|
||||
}},
|
||||
"mapping": {{
|
||||
"{ex1}": "左袖",
|
||||
"{ex2}": "前片"
|
||||
}},
|
||||
"analysis": [
|
||||
{{"layer": "{ex1}", "piece": "左袖", "type": "all_over", "visible": true, "description": "黑底白花渐变"}},
|
||||
{{"layer": "{ex2}", "piece": "前片", "type": "all_over", "visible": true, "description": "上半黑底大花+下半灰白格纹"}},
|
||||
{{"layer": "{ex3}", "piece": "后片", "type": "all_over", "visible": false, "inferred_from": "{ex2}", "inference_reason": "全身花型面料，后片花型与前片相同", "confidence": "high"}}
|
||||
],
|
||||
"symmetric_pairs": [["{ex1}", "对称片名"]],
|
||||
"base_color": "#000000"
|
||||
}}
|
||||
```
|
||||
|
||||
⚠️ 重要：
|
||||
- mapping 和 analysis 中的 layer 必须用**实际图层组名称**
|
||||
- 看不到的部位设 visible: false 并说明推理依据
|
||||
- 左右对称的片标记 symmetric_pairs
|
||||
只返回 JSON，不要其他文字。"""
|
||||
else:
|
||||
prompt = f"""这是一张 PS 文档裁片截图。图层信息：
|
||||
{layers_info}
|
||||
|
||||
识别每个图层组的服装部位，mapping 的 key 必须用**实际图层名**：
|
||||
```json
|
||||
{{"mapping": {{"{ex1}": "前片", "{ex2}": "后片"}}}}
|
||||
```
|
||||
只返回 JSON。"""
|
||||
|
||||
log.info(f"[IdentifyPieces] 使用豆包视觉: {use_model}")
|
||||
images = []
|
||||
if garment_b64:
|
||||
images.append(garment_b64)
|
||||
images.append(canvas_b64)
|
||||
content = call_vision_messages(
|
||||
prompt,
|
||||
images,
|
||||
model_override=use_model,
|
||||
runtime_config=runtime_ai_config,
|
||||
)
|
||||
log.info(f"[IdentifyPieces] 视觉模型回复: {content[:500]}")
|
||||
|
||||
try:
|
||||
clean = content.strip()
|
||||
if "```" in clean:
|
||||
clean = clean.split("```")[1]
|
||||
if clean.startswith("json"):
|
||||
clean = clean[4:]
|
||||
clean = clean.strip()
|
||||
parsed = json.loads(clean)
|
||||
|
||||
if "mapping" in parsed:
|
||||
return parsed
|
||||
else:
|
||||
return {"mapping": parsed}
|
||||
except json.JSONDecodeError:
|
||||
log.warning(f"[IdentifyPieces] JSON 解析失败")
|
||||
return {"mapping": {}, "raw_response": content}
|
||||
@@ -1,17 +1,32 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
AI 模型调用层
|
||||
统一管理所有 LLM / Vision / Image 模型的调用逻辑
|
||||
支持 Qwen (DashScope) 和 Gemini (第三方代理) 两套路由
|
||||
统一管理所有 AI 模型的调用逻辑
|
||||
当前默认 provider 为 Ark，配置项保持通用命名，并支持技能化执行策略
|
||||
"""
|
||||
|
||||
from typing import List
|
||||
import json, base64, re, logging
|
||||
from typing import List, Optional
|
||||
from dataclasses import dataclass
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from app.core.config import settings
|
||||
from app.api.v1.ai_tools import PS_TOOLS, TOOL_DISPLAY_NAMES
|
||||
from app.api.v1.ai_skills import AiSkill, get_ai_skill, resolve_ai_skill
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class RuntimeAiConfig:
|
||||
provider: str
|
||||
api_key: str
|
||||
base_url: str
|
||||
chat_base_url: str = ""
|
||||
vision_base_url: str = ""
|
||||
image_base_url: str = ""
|
||||
source: str = "server"
|
||||
|
||||
# ==================== Prompts ====================
|
||||
|
||||
SYSTEM_PROMPT = """你是 DesignerCEP 的 AI 助手，运行在 Adobe Photoshop CEP 插件中。
|
||||
@@ -20,395 +35,634 @@ SYSTEM_PROMPT = """你是 DesignerCEP 的 AI 助手，运行在 Adobe Photoshop
|
||||
1. 回答关于 Photoshop 操作和插件使用的问题
|
||||
2. 通过工具直接操作 Photoshop（创建图层、对齐、查看文档信息等）
|
||||
3. 帮用户排查操作中遇到的错误
|
||||
4. **AI 智能套图**：两阶段流程 — 先生成预览确认，再提取套到裁片上
|
||||
4. AI 智能套图：两阶段流程，先生成预览确认，再提取套到裁片上
|
||||
|
||||
## AI 智能套图流程（两阶段）
|
||||
## AI 智能套图流程
|
||||
|
||||
当用户上传成衣图片并要求套图时：
|
||||
|
||||
### 阶段 1 — 识别裁片 + 生成预览（需用户确认）
|
||||
1. 调用 **identify_pieces** — 截取画布并识别每个图层是什么裁片部位（前片、后片、袖子等）
|
||||
2. 告诉用户识别结果（如 "M-1=前片, M-2=后片, M-5=左袖..."）
|
||||
3. 调用 **generate_garment_preview** — 会自动在裁片下方标注名称标签（如 "M-1 前片"），然后截取带标签的画布 + 成衣照片一起发给 AI 生成预览
|
||||
4. 预览图显示在聊天中，每个裁片都有标签，**等用户确认 OK 后**进入阶段 2
|
||||
|
||||
### 阶段 2 — 提取花样 + 正式套图（用户确认后）
|
||||
5. 根据 identify_pieces 分析结果中每个裁片的 type 决定处理方式：
|
||||
- solid → 设 color 字段，PS 直接纯色填充
|
||||
- fill_pattern → AI 提取花型铺满
|
||||
- theme_pattern → 底层 PS 纯色填充（设 color）+ 上层 AI 提取主题图案（白底+正片叠底）
|
||||
- mixed_pattern → 底层 AI 提取花型 + 上层 AI 提取主题图案（白底+正片叠底）
|
||||
6. 调用 **extract_and_apply_all_pieces** 执行套图
|
||||
7. 可选：调用 **verify_pattern_result** 验证效果
|
||||
1. 先调用 identify_pieces 识别每个图层是什么裁片部位
|
||||
2. 告诉用户识别结果
|
||||
3. 调用 generate_garment_preview 生成带标签的裁片预览图
|
||||
4. 等用户确认 OK 后，再调用 extract_and_apply_all_pieces 正式套图
|
||||
|
||||
重要：
|
||||
- 阶段 1 完成后必须**等用户说"可以"/"OK"**才执行阶段 2
|
||||
- 用户可能会说"袖子纯色"、"后幅不要花样"等，要根据 identify_pieces 的结果对应到正确的图层名
|
||||
|
||||
重要规则：
|
||||
- 当用户要求执行 PS 操作时，使用工具完成
|
||||
- 执行操作前可以先了解当前文档和图层状态
|
||||
- 用简洁的中文回答，适合在小面板中阅读
|
||||
- 如果工具执行失败，向用户解释原因并建议解决方案
|
||||
- 预览生成后必须等用户确认，不能自己直接进入正式套图
|
||||
- 每个工具最多调用 1 次，除非上次执行失败
|
||||
- 用户要求执行 Photoshop 操作时，优先使用工具完成
|
||||
- 用户如果是在问“不会怎么做 / 为什么不行 / 下一步怎么操作”，要优先结合当前 Photoshop 上下文答疑，不要只给脱离现场的教程
|
||||
- 答疑、查询当前信息、执行操作可以在同一轮里完成，不要把它们人为拆成两种模式
|
||||
- 如果用户的问题和当前文档、图层、选区、参考图有关，优先先查 1 个最相关的上下文工具，再给结论
|
||||
- 如果用户意图已经明确且风险低，可以边解释边执行；如果涉及删除、覆盖、关闭、合并等风险操作，先确认
|
||||
- 不要只讲原理不动手，也不要只闷头执行不解释；默认输出“判断 + 当前检查/执行结果 + 下一步”
|
||||
- 用简洁中文回答，适合在小面板中阅读
|
||||
- 复杂任务先给 2-4 条简短思路，再开始执行
|
||||
- 默认按 观察 -> 判断 -> 执行 -> 验证 的顺序做事
|
||||
- 如果任务包含多步骤，不要一上来同时调很多工具，先完成当前阶段再进入下一阶段
|
||||
- 回答里尽量显式说明“当前阶段”和“下一步”
|
||||
"""
|
||||
|
||||
VISION_PROMPT = """你是一位资深的服装设计分析师，同时也是 DesignerCEP 的 AI 助手。
|
||||
|
||||
当用户发送服装/成衣图片时，请从以下维度进行专业分析：
|
||||
|
||||
1. **服装类别** — 上衣/裤子/裙子/连衣裙/外套/配饰等，细分款式
|
||||
2. **面料分析** — 根据视觉特征推测面料类型（棉、涤纶、丝绸、针织、牛仔、雪纺等），分析面料质感
|
||||
3. **颜色与印花** — 主色调、配色方案、印花/图案类型及工艺（数码印花、丝网印刷、提花等）
|
||||
4. **版型特点** — 修身/宽松/A字/H型等，分析领口、袖型、肩线、腰线、下摆处理
|
||||
5. **工艺细节** — 缝线工艺、拉链/纽扣/暗扣、口袋设计、装饰细节、包边/锁边
|
||||
6. **设计评价** — 设计亮点、风格定位（休闲/正装/运动/时尚等）、目标消费群体
|
||||
7. **改进建议** — 如有可改进之处，给出专业建议
|
||||
请结合图片和用户问题，从服装类别、面料、颜色印花、版型特点、工艺细节、设计评价、改进建议等维度做专业分析。
|
||||
|
||||
规则：
|
||||
- 如果图片不是服装相关，也请尽力分析图片内容并给出有价值的反馈
|
||||
- 如果用户同时提了文字问题，请结合图片和问题一起回答
|
||||
- 用清晰、结构化的中文回答，适合在设计工作中参考
|
||||
- 回答要专业但不啰嗦，突出重点信息
|
||||
- 如果图片不是服装相关，也尽力分析图片内容
|
||||
- 用清晰、结构化的中文回答
|
||||
- 回答要专业但不啰嗦，突出重点
|
||||
"""
|
||||
|
||||
TOOL_BY_NAME = {
|
||||
tool["function"]["name"]: tool
|
||||
for tool in PS_TOOLS
|
||||
if tool.get("type") == "function" and tool.get("function", {}).get("name")
|
||||
}
|
||||
|
||||
|
||||
# ==================== 客户端工厂 ====================
|
||||
|
||||
|
||||
def get_runtime_ai_config(user=None) -> RuntimeAiConfig:
|
||||
user_provider = (getattr(user, "ai_provider", "") or "").strip().lower()
|
||||
user_api_key = (getattr(user, "ai_api_key", "") or "").strip()
|
||||
user_base_url = (getattr(user, "ai_base_url", "") or "").strip()
|
||||
user_chat_base_url = (getattr(user, "ai_chat_base_url", "") or "").strip()
|
||||
user_vision_base_url = (getattr(user, "ai_vision_base_url", "") or "").strip()
|
||||
user_image_base_url = (getattr(user, "ai_image_base_url", "") or "").strip()
|
||||
|
||||
default_base_url = (
|
||||
settings.AI_BASE_URL
|
||||
or settings.ARK_BASE_URL
|
||||
or "https://ark.cn-beijing.volces.com/api/v3"
|
||||
)
|
||||
|
||||
if user_api_key:
|
||||
return RuntimeAiConfig(
|
||||
provider=user_provider or (settings.AI_PROVIDER or "ark").strip().lower(),
|
||||
api_key=user_api_key,
|
||||
base_url=user_base_url or default_base_url,
|
||||
chat_base_url=user_chat_base_url or settings.AI_CHAT_BASE_URL or "",
|
||||
vision_base_url=user_vision_base_url or settings.AI_VISION_BASE_URL or "",
|
||||
image_base_url=user_image_base_url or settings.AI_IMAGE_BASE_URL or "",
|
||||
source="user",
|
||||
)
|
||||
|
||||
return RuntimeAiConfig(
|
||||
provider=(settings.AI_PROVIDER or "ark").strip().lower(),
|
||||
api_key=(
|
||||
settings.AI_API_KEY
|
||||
or settings.ARK_API_KEY
|
||||
or os.getenv("AI_API_KEY", "")
|
||||
or os.getenv("ARK_API_KEY", "")
|
||||
),
|
||||
base_url=default_base_url,
|
||||
chat_base_url=settings.AI_CHAT_BASE_URL or "",
|
||||
vision_base_url=settings.AI_VISION_BASE_URL or "",
|
||||
image_base_url=settings.AI_IMAGE_BASE_URL or "",
|
||||
source="server",
|
||||
)
|
||||
|
||||
def get_ai_provider(runtime_config: Optional[RuntimeAiConfig] = None) -> str:
|
||||
return (runtime_config.provider if runtime_config else settings.AI_PROVIDER or "ark").strip().lower()
|
||||
|
||||
|
||||
def get_ai_api_key(runtime_config: Optional[RuntimeAiConfig] = None) -> str:
|
||||
return (runtime_config.api_key if runtime_config else get_runtime_ai_config().api_key).strip()
|
||||
|
||||
|
||||
def get_ai_base_url(
|
||||
runtime_config: Optional[RuntimeAiConfig] = None, capability: str = "chat"
|
||||
) -> str:
|
||||
config = runtime_config or get_runtime_ai_config()
|
||||
if capability == "vision":
|
||||
return (config.vision_base_url or config.base_url).strip()
|
||||
if capability == "image":
|
||||
return (config.image_base_url or config.base_url).strip()
|
||||
return (config.chat_base_url or config.base_url).strip()
|
||||
|
||||
|
||||
def has_ai_config(runtime_config: Optional[RuntimeAiConfig] = None) -> bool:
|
||||
return bool(get_ai_api_key(runtime_config))
|
||||
|
||||
|
||||
def get_ai_client(
|
||||
runtime_config: Optional[RuntimeAiConfig] = None, capability: str = "chat"
|
||||
):
|
||||
"""获取当前 provider 的客户端。"""
|
||||
from volcenginesdkarkruntime import Ark
|
||||
|
||||
provider = get_ai_provider(runtime_config)
|
||||
if provider != "ark":
|
||||
raise ValueError(f"当前暂不支持的 AI_PROVIDER: {provider}")
|
||||
|
||||
api_key = get_ai_api_key(runtime_config)
|
||||
if not api_key:
|
||||
raise ValueError("AI_API_KEY 未配置")
|
||||
|
||||
return Ark(
|
||||
base_url=get_ai_base_url(runtime_config, capability=capability),
|
||||
api_key=api_key,
|
||||
)
|
||||
|
||||
|
||||
def is_gemini_model(model_name: str) -> bool:
|
||||
"""兼容旧代码，当前已统一走豆包。"""
|
||||
return False
|
||||
|
||||
|
||||
def _tool_schemas_for_skill(skill: AiSkill) -> List[dict]:
|
||||
if not skill.allowed_tools:
|
||||
return []
|
||||
return [TOOL_BY_NAME[name] for name in skill.allowed_tools if name in TOOL_BY_NAME]
|
||||
|
||||
|
||||
def _numbered_block(items: tuple[str, ...]) -> str:
|
||||
if not items:
|
||||
return "无"
|
||||
return "\n".join(f"{idx + 1}. {item}" for idx, item in enumerate(items))
|
||||
|
||||
|
||||
def _bullet_block(items: tuple[str, ...]) -> str:
|
||||
if not items:
|
||||
return "无"
|
||||
return "\n".join(f"- {item}" for item in items)
|
||||
|
||||
|
||||
def _build_skill_prompt(skill: AiSkill, has_image: bool = False) -> str:
|
||||
workflow = _numbered_block(skill.workflow)
|
||||
guardrails = _bullet_block(skill.guardrails)
|
||||
success_criteria = _bullet_block(skill.success_criteria)
|
||||
triage_questions = _numbered_block(skill.triage_questions)
|
||||
deliverables = _bullet_block(skill.deliverables)
|
||||
execution_notes = _bullet_block(skill.execution_notes)
|
||||
tool_names = (
|
||||
"、".join(TOOL_DISPLAY_NAMES.get(name, name) for name in skill.allowed_tools)
|
||||
if skill.allowed_tools
|
||||
else "当前技能以分析和建议为主，不主动调用 Photoshop 工具。"
|
||||
)
|
||||
origin = skill.origin or "项目内置技能"
|
||||
origin_url = skill.origin_url or "无"
|
||||
upstream_skill = skill.upstream_skill or "无"
|
||||
|
||||
return f"""当前启用技能：{skill.name}（{skill.id}）
|
||||
技能定位：{skill.description}
|
||||
执行模式：{skill.mode}
|
||||
规划风格：{skill.planning_style}
|
||||
图片提示：{skill.image_hint or '无'}
|
||||
来源：{origin}
|
||||
上游技能：{upstream_skill}
|
||||
来源地址：{origin_url}
|
||||
|
||||
推荐工作流：
|
||||
{workflow}
|
||||
|
||||
预判问题：
|
||||
{triage_questions}
|
||||
|
||||
期望交付：
|
||||
{deliverables}
|
||||
|
||||
执行边界：
|
||||
{guardrails}
|
||||
|
||||
执行备注：
|
||||
{execution_notes}
|
||||
|
||||
成功标准：
|
||||
{success_criteria}
|
||||
|
||||
本技能优先工具：
|
||||
{tool_names}
|
||||
|
||||
本轮补充要求：
|
||||
- 回复保持中文、简洁、能落地。
|
||||
- 涉及具体执行时，先根据当前技能给出简短思路，再调用工具。
|
||||
- 根据用户问题动态混合答疑、查询与执行；如果用户是在求助“怎么做/为什么失败/下一步怎么弄”，不要把答疑和执行拆成两轮。
|
||||
- 能读取当前 Photoshop 上下文时，优先读当前信息后再回答；不要只给泛化教程。
|
||||
- 优先先想清楚计划，再动手执行；如果任务复杂，先向用户同步 2 到 4 条阶段计划。
|
||||
- 每轮优先调用最相关的 1 个工具，避免一口气乱调很多工具。
|
||||
- 如果图片信息不足、文档上下文不足或风险较高，要先说明再操作。
|
||||
- 每完成一个阶段，要用一句中文总结已完成内容，并指向下一步。
|
||||
- {'本轮带有图片，请结合图片内容和技能策略做判断。' if has_image else '本轮没有新图片，优先结合聊天历史和 Photoshop 上下文。'}
|
||||
"""
|
||||
|
||||
|
||||
# ==================== 路由判断 ====================
|
||||
|
||||
def is_gemini_model(model_name: str) -> bool:
|
||||
"""判断是否是 Gemini 模型"""
|
||||
return bool(model_name) and "gemini" in model_name.lower()
|
||||
def _chat_system_prompt(skill: AiSkill, has_image: bool = False) -> str:
|
||||
return f"{SYSTEM_PROMPT}\n\n{_build_skill_prompt(skill, has_image)}"
|
||||
|
||||
|
||||
# ==================== Qwen / OpenAI 兼容调用 ====================
|
||||
def _vision_instructions(skill: AiSkill) -> str:
|
||||
return f"{VISION_PROMPT}\n\n{_build_skill_prompt(skill, has_image=True)}"
|
||||
|
||||
def call_llm_with_tools(messages_history: List[dict], model_override: str = None):
|
||||
"""调用 LLM，支持 function calling"""
|
||||
from openai import OpenAI
|
||||
|
||||
def _messages_with_system(
|
||||
messages_history: List[dict], skill: AiSkill, has_image: bool = False
|
||||
) -> List[dict]:
|
||||
return [{"role": "system", "content": _chat_system_prompt(skill, has_image)}] + messages_history
|
||||
|
||||
|
||||
def _extract_response_text(response) -> str:
|
||||
texts: List[str] = []
|
||||
for output_item in getattr(response, "output", []) or []:
|
||||
for content_item in getattr(output_item, "content", []) or []:
|
||||
if getattr(content_item, "type", "") == "output_text":
|
||||
text_value = getattr(content_item, "text", "")
|
||||
if text_value:
|
||||
texts.append(text_value)
|
||||
return "\n".join(texts).strip()
|
||||
|
||||
|
||||
def _image_data_url(image_base64: str) -> str:
|
||||
return f"data:image/jpeg;base64,{image_base64}"
|
||||
|
||||
|
||||
# ==================== 聊天 / 工具调用 ====================
|
||||
|
||||
|
||||
def call_llm_with_tools(
|
||||
messages_history: List[dict],
|
||||
model_override: str = None,
|
||||
skill_id: Optional[str] = None,
|
||||
has_image: bool = False,
|
||||
runtime_config: Optional[RuntimeAiConfig] = None,
|
||||
):
|
||||
"""调用当前配置的对话模型，支持 function calling。"""
|
||||
use_model = model_override or settings.AI_MODEL
|
||||
|
||||
client = OpenAI(
|
||||
api_key=settings.AI_API_KEY,
|
||||
base_url=settings.AI_BASE_URL or "https://api.openai.com/v1",
|
||||
client = get_ai_client(runtime_config, capability="chat")
|
||||
skill = get_ai_skill(skill_id) or resolve_ai_skill(
|
||||
message=str(messages_history[-1].get("content", "")) if messages_history else "",
|
||||
history_messages=messages_history[:-1],
|
||||
requested_skill_id=skill_id,
|
||||
has_image=has_image,
|
||||
)
|
||||
messages = _messages_with_system(messages_history, skill, has_image)
|
||||
tools = _tool_schemas_for_skill(skill)
|
||||
|
||||
messages = [{"role": "system", "content": SYSTEM_PROMPT}] + messages_history
|
||||
log.info(f"{'=' * 60}")
|
||||
log.info(f"[LLM] provider={get_ai_provider(runtime_config)} model={use_model} source={(runtime_config.source if runtime_config else 'server')}")
|
||||
log.info(f"[LLM] skill={skill.id}")
|
||||
log.info(f"[LLM] 消息数量: {len(messages)}")
|
||||
log.info(f"[LLM] 工具数量: {len(tools)}")
|
||||
|
||||
log.info(f"{'='*60}")
|
||||
log.info(f"[LLM] 调用工具模型: {use_model}{' (override)' if model_override else ''}")
|
||||
log.info(f"[LLM] 消息数量: {len(messages)} (system + {len(messages_history)} history)")
|
||||
for i, m in enumerate(messages_history[-5:]):
|
||||
role = m['role']
|
||||
content = m['content'][:120] if m.get('content') else '(empty)'
|
||||
log.info(f"[LLM] history[-{len(messages_history)-i}] {role}: {content}")
|
||||
log.info(f"[LLM] 工具数量: {len(PS_TOOLS)}")
|
||||
request_kwargs = {
|
||||
"model": use_model,
|
||||
"messages": messages,
|
||||
"temperature": 0.3,
|
||||
}
|
||||
if tools:
|
||||
request_kwargs.update(
|
||||
{
|
||||
"tools": tools,
|
||||
"tool_choice": "auto",
|
||||
"parallel_tool_calls": False,
|
||||
}
|
||||
)
|
||||
|
||||
completion = client.chat.completions.create(
|
||||
model=use_model,
|
||||
messages=messages,
|
||||
tools=PS_TOOLS,
|
||||
tool_choice="auto",
|
||||
)
|
||||
completion = client.chat.completions.create(**request_kwargs)
|
||||
|
||||
choice = completion.choices[0]
|
||||
message = choice.message
|
||||
tool_calls = getattr(message, "tool_calls", None) or []
|
||||
|
||||
log.info(f"[LLM] 响应 finish_reason={choice.finish_reason}")
|
||||
if message.content:
|
||||
log.info(f"[LLM] 回复文本: {message.content[:150]}...")
|
||||
if message.tool_calls:
|
||||
for tc in message.tool_calls:
|
||||
log.info(f"[LLM] 工具调用: {tc.function.name}({tc.function.arguments[:200]})")
|
||||
else:
|
||||
log.info(f"[LLM] 无工具调用")
|
||||
log.info(f"{'='*60}")
|
||||
|
||||
if message.tool_calls and len(message.tool_calls) > 0:
|
||||
if tool_calls:
|
||||
tool_calls_data = []
|
||||
for tc in message.tool_calls:
|
||||
for tc in tool_calls:
|
||||
args = {}
|
||||
if tc.function.arguments:
|
||||
function = getattr(tc, "function", None)
|
||||
raw_args = getattr(function, "arguments", "") if function else ""
|
||||
if raw_args:
|
||||
try:
|
||||
args = json.loads(tc.function.arguments)
|
||||
args = json.loads(raw_args)
|
||||
except json.JSONDecodeError:
|
||||
args = {}
|
||||
tool_calls_data.append({
|
||||
"id": tc.id,
|
||||
"name": tc.function.name,
|
||||
"display_name": TOOL_DISPLAY_NAMES.get(tc.function.name, tc.function.name),
|
||||
"args": args,
|
||||
"status": "pending"
|
||||
})
|
||||
return message.content or "", tool_calls_data
|
||||
function_name = getattr(function, "name", "") if function else ""
|
||||
tool_calls_data.append(
|
||||
{
|
||||
"id": getattr(tc, "id", function_name),
|
||||
"name": function_name,
|
||||
"display_name": TOOL_DISPLAY_NAMES.get(function_name, function_name),
|
||||
"args": args,
|
||||
"status": "pending",
|
||||
}
|
||||
)
|
||||
return getattr(message, "content", "") or "", tool_calls_data, skill.to_public_dict()
|
||||
|
||||
return message.content or "", None
|
||||
return getattr(message, "content", "") or "", None, skill.to_public_dict()
|
||||
|
||||
|
||||
# ==================== Gemini 调用（OpenAI 兼容代理） ====================
|
||||
|
||||
def call_gemini(messages_history: List[dict], model: str, images_b64: List[str] = None) -> str:
|
||||
"""调用 Gemini（通过第三方代理，OpenAI 兼容格式）"""
|
||||
from openai import OpenAI as _OpenAI
|
||||
|
||||
if not settings.GEMINI_API_KEY or not settings.GEMINI_BASE_URL:
|
||||
raise ValueError("GEMINI_API_KEY 或 GEMINI_BASE_URL 未配置")
|
||||
|
||||
client = _OpenAI(
|
||||
api_key=settings.GEMINI_API_KEY,
|
||||
base_url=f"{settings.GEMINI_BASE_URL}/v1",
|
||||
)
|
||||
|
||||
messages = []
|
||||
def call_gemini(messages_history: List[dict], model: str, images_b64: List[str] = None):
|
||||
"""兼容旧接口，内部统一转到当前视觉模型。"""
|
||||
prompt = ""
|
||||
for msg in messages_history:
|
||||
role = msg.get("role", "user")
|
||||
content = msg.get("content", "")
|
||||
if role not in ("system", "user", "assistant"):
|
||||
role = "user"
|
||||
messages.append({"role": role, "content": content})
|
||||
|
||||
if msg.get("role") == "user" and msg.get("content"):
|
||||
prompt = str(msg["content"])
|
||||
if images_b64:
|
||||
last_user_idx = None
|
||||
for i in range(len(messages) - 1, -1, -1):
|
||||
if messages[i]["role"] == "user":
|
||||
last_user_idx = i
|
||||
break
|
||||
if last_user_idx is not None:
|
||||
text_content = messages[last_user_idx]["content"]
|
||||
multimodal_content = []
|
||||
for img_b64 in images_b64:
|
||||
multimodal_content.append({
|
||||
"type": "image_url",
|
||||
"image_url": {"url": f"data:image/jpeg;base64,{img_b64}"}
|
||||
})
|
||||
multimodal_content.append({"type": "text", "text": text_content})
|
||||
messages[last_user_idx]["content"] = multimodal_content
|
||||
return call_vision_messages(prompt or "请分析这张图片。", images_b64, model)
|
||||
|
||||
log.info(f"{'='*60}")
|
||||
log.info(f"[Gemini] 调用模型: {model} (OpenAI 兼容)")
|
||||
log.info(f"[Gemini] 消息数: {len(messages)}, 图片数: {len(images_b64) if images_b64 else 0}")
|
||||
|
||||
completion = client.chat.completions.create(model=model, messages=messages)
|
||||
result = completion.choices[0].message.content or ""
|
||||
log.info(f"[Gemini] 回复: {result[:200]}...")
|
||||
return result
|
||||
reply, _, _ = call_llm_with_tools(messages_history, model_override=model)
|
||||
return reply
|
||||
|
||||
|
||||
def call_gemini_with_tools(messages_history: List[dict], model: str) -> tuple:
|
||||
"""用 Gemini 做对话（不支持 function calling）"""
|
||||
full_history = [{"role": "system", "content": SYSTEM_PROMPT}] + messages_history
|
||||
|
||||
log.info(f"{'='*60}")
|
||||
log.info(f"[Gemini] 调用对话模型: {model}")
|
||||
for i, m in enumerate(messages_history[-3:]):
|
||||
log.info(f"[Gemini] history[-{len(messages_history)-i}] {m['role']}: {str(m.get('content',''))[:120]}")
|
||||
|
||||
result = call_gemini(full_history, model)
|
||||
log.info(f"[Gemini] 回复文本: {result[:150]}...")
|
||||
return result, None
|
||||
"""兼容旧接口，内部统一转到当前工具调用。"""
|
||||
return call_llm_with_tools(messages_history, model_override=model)
|
||||
|
||||
|
||||
# ==================== 视觉模型 ====================
|
||||
|
||||
def call_vision_llm(user_message: str, image_base64: str, history: List[dict], model_override: str = None) -> str:
|
||||
"""调用视觉模型分析图片（自动路由 Qwen / Gemini）"""
|
||||
|
||||
def call_vision_messages(
|
||||
user_message: str,
|
||||
images_b64: List[str],
|
||||
model_override: str = None,
|
||||
instructions_override: Optional[str] = None,
|
||||
runtime_config: Optional[RuntimeAiConfig] = None,
|
||||
) -> str:
|
||||
"""调用当前视觉模型分析图片。"""
|
||||
use_model = model_override or settings.AI_VISION_MODEL
|
||||
client = get_ai_client(runtime_config, capability="vision")
|
||||
|
||||
if is_gemini_model(use_model):
|
||||
log.info(f"[Vision] 使用 Gemini 视觉模型: {use_model}")
|
||||
msgs = [{"role": "system", "content": VISION_PROMPT}]
|
||||
for h in history[-10:]:
|
||||
role = h["role"] if h["role"] != "tool" else "user"
|
||||
msgs.append({"role": role, "content": h["content"]})
|
||||
msgs.append({"role": "user", "content": user_message})
|
||||
return call_gemini(msgs, use_model, images_b64=[image_base64])
|
||||
content = []
|
||||
for image_b64 in images_b64:
|
||||
content.append({"type": "input_image", "image_url": _image_data_url(image_b64)})
|
||||
content.append({"type": "input_text", "text": user_message})
|
||||
|
||||
from openai import OpenAI
|
||||
client = OpenAI(api_key=settings.AI_API_KEY, base_url=settings.AI_BASE_URL or "https://api.openai.com/v1")
|
||||
response = client.responses.create(
|
||||
model=use_model,
|
||||
instructions=instructions_override or VISION_PROMPT,
|
||||
input=[{"role": "user", "content": content}],
|
||||
)
|
||||
|
||||
user_content = [
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}},
|
||||
{"type": "text", "text": user_message}
|
||||
]
|
||||
messages = [{"role": "system", "content": VISION_PROMPT}]
|
||||
for h in history[-10:]:
|
||||
role = h["role"] if h["role"] != "tool" else "user"
|
||||
messages.append({"role": role, "content": h["content"]})
|
||||
messages.append({"role": "user", "content": user_content})
|
||||
|
||||
completion = client.chat.completions.create(model=use_model, messages=messages)
|
||||
return completion.choices[0].message.content or ""
|
||||
result = _extract_response_text(response)
|
||||
log.info(
|
||||
f"[Vision] provider={get_ai_provider(runtime_config)} model={use_model} 输出长度: {len(result)}"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
# ==================== 图片编辑/生成模型 ====================
|
||||
def call_vision_llm(
|
||||
user_message: str,
|
||||
image_base64: str,
|
||||
history: List[dict],
|
||||
model_override: str = None,
|
||||
skill_id: Optional[str] = None,
|
||||
runtime_config: Optional[RuntimeAiConfig] = None,
|
||||
) -> str:
|
||||
"""兼容旧接口，调用当前视觉模型。"""
|
||||
skill = get_ai_skill(skill_id) or resolve_ai_skill(
|
||||
message=user_message,
|
||||
history_messages=history,
|
||||
requested_skill_id=skill_id,
|
||||
has_image=True,
|
||||
)
|
||||
combined_message = user_message
|
||||
if history:
|
||||
recent_text = "\n".join(
|
||||
str(item.get("content", "")) for item in history[-5:] if item.get("content")
|
||||
)
|
||||
if recent_text:
|
||||
combined_message = f"历史对话：\n{recent_text}\n\n当前问题：{user_message}"
|
||||
return call_vision_messages(
|
||||
combined_message,
|
||||
[image_base64],
|
||||
model_override,
|
||||
instructions_override=_vision_instructions(skill),
|
||||
runtime_config=runtime_config,
|
||||
)
|
||||
|
||||
def call_image_model(images_b64: List[str], prompt: str, model_override: str = None) -> tuple:
|
||||
"""调用图片编辑/生成模型（自动路由 DashScope / Gemini），返回 (url_or_datauri, description)"""
|
||||
import requests as http_requests
|
||||
|
||||
# ==================== 图片编辑 / 生成 ====================
|
||||
|
||||
|
||||
def call_image_model(
|
||||
images_b64: List[str],
|
||||
prompt: str,
|
||||
model_override: str = None,
|
||||
runtime_config: Optional[RuntimeAiConfig] = None,
|
||||
) -> tuple:
|
||||
"""调用当前图片模型，返回 (url_or_datauri, description)。"""
|
||||
result = call_image_model_batch(
|
||||
images_b64=images_b64,
|
||||
prompt=prompt,
|
||||
model_override=model_override,
|
||||
size="2K",
|
||||
max_images=1,
|
||||
stream=False,
|
||||
runtime_config=runtime_config,
|
||||
)
|
||||
first_image = result["images"][0]
|
||||
return str(first_image["url"]), ""
|
||||
|
||||
|
||||
def _image_response_error_message(payload) -> str:
|
||||
error = getattr(payload, "error", None)
|
||||
return str(getattr(error, "message", "") or "").strip()
|
||||
|
||||
|
||||
def _image_usage_dict(payload) -> Optional[dict]:
|
||||
usage = getattr(payload, "usage", None)
|
||||
if usage is None:
|
||||
return None
|
||||
if hasattr(usage, "model_dump"):
|
||||
return usage.model_dump(mode="json")
|
||||
if isinstance(usage, dict):
|
||||
return usage
|
||||
return None
|
||||
|
||||
|
||||
def _image_item_to_result(image_item, index: int) -> dict:
|
||||
image_url = getattr(image_item, "url", "") or ""
|
||||
image_b64 = getattr(image_item, "b64_json", "") or ""
|
||||
image_size = getattr(image_item, "size", "") or ""
|
||||
|
||||
if image_url:
|
||||
return {"index": index, "url": image_url, "size": image_size}
|
||||
|
||||
if image_b64:
|
||||
padding = (-len(image_b64)) % 4
|
||||
if padding:
|
||||
image_b64 += "=" * padding
|
||||
return {
|
||||
"index": index,
|
||||
"url": f"data:image/png;base64,{image_b64}",
|
||||
"size": image_size,
|
||||
}
|
||||
|
||||
raise ValueError("图片模型返回了空图片结果")
|
||||
|
||||
|
||||
def call_image_model_batch(
|
||||
images_b64: List[str],
|
||||
prompt: str,
|
||||
model_override: str = None,
|
||||
size: str = "2K",
|
||||
max_images: int = 1,
|
||||
stream: Optional[bool] = None,
|
||||
runtime_config: Optional[RuntimeAiConfig] = None,
|
||||
) -> dict:
|
||||
"""调用当前图片模型，支持单图、参考图生图和连续组图。"""
|
||||
from volcenginesdkarkruntime.types.images.images import (
|
||||
SequentialImageGenerationOptions,
|
||||
)
|
||||
|
||||
use_model = model_override or settings.AI_IMAGE_EDIT_MODEL
|
||||
client = get_ai_client(runtime_config, capability="image")
|
||||
image_count = max(1, min(int(max_images or 1), 4))
|
||||
should_stream = (image_count > 1) if stream is None else bool(stream)
|
||||
|
||||
# ---------- Gemini（OpenAI 兼容代理） ----------
|
||||
if is_gemini_model(use_model):
|
||||
log.info(f"[ImageModel] 使用 Gemini 图片模型: {use_model}")
|
||||
if not settings.GEMINI_API_KEY or not settings.GEMINI_BASE_URL:
|
||||
raise ValueError("GEMINI_API_KEY 或 GEMINI_BASE_URL 未配置")
|
||||
|
||||
from openai import OpenAI as _OpenAI
|
||||
client = _OpenAI(api_key=settings.GEMINI_API_KEY, base_url=f"{settings.GEMINI_BASE_URL}/v1")
|
||||
|
||||
content_parts = [{"type": "text", "text": prompt}]
|
||||
for img_b64 in images_b64:
|
||||
content_parts.append({"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{img_b64}"}})
|
||||
|
||||
log.info(f"[ImageModel] Gemini OpenAI 兼容, 模型: {use_model}")
|
||||
log.info(f"[ImageModel] 图片数: {len(images_b64)}, 各图: {[len(b)//1024 for b in images_b64]}KB")
|
||||
log.info(f"[ImageModel] 提示词: {prompt[:200]}")
|
||||
|
||||
completion = client.chat.completions.create(model=use_model, messages=[{"role": "user", "content": content_parts}])
|
||||
result_content = completion.choices[0].message.content or ""
|
||||
log.info(f"[ImageModel] Gemini 回复长度: {len(result_content)} chars")
|
||||
|
||||
# 提取 base64 图片
|
||||
match = re.search(r'!\[.*?\]\((data:image/(\w+);base64,([^)]+))\)', result_content)
|
||||
if not match:
|
||||
match = re.search(r'(data:image/(\w+);base64,([A-Za-z0-9+/=]+))', result_content)
|
||||
if not match:
|
||||
log.warning(f"[ImageModel] Gemini 响应中无图片，前500字: {result_content[:500]}")
|
||||
raise ValueError("Gemini 未返回图片，请检查模型是否支持图片生成")
|
||||
|
||||
img_format = match.group(2)
|
||||
image_b64 = match.group(3)
|
||||
padding = 4 - len(image_b64) % 4
|
||||
if padding != 4:
|
||||
image_b64 += '=' * padding
|
||||
|
||||
log.info(f"[ImageModel] Gemini 返回图片: image/{img_format}, {len(image_b64)//1024}KB")
|
||||
_save_debug_image(base64.b64decode(image_b64), f'gemini_output.{img_format}')
|
||||
|
||||
description = re.sub(r'!\[.*?\]\(data:image/[^)]+\)', '', result_content).strip()
|
||||
return f"data:image/{img_format};base64,{image_b64}", description
|
||||
|
||||
# ---------- DashScope 原生接口 ----------
|
||||
api_url = "https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation"
|
||||
content_parts = []
|
||||
for img_b64 in images_b64:
|
||||
content_parts.append({"image": f"data:image/jpeg;base64,{img_b64}"})
|
||||
content_parts.append({"text": prompt})
|
||||
|
||||
payload = {
|
||||
"model": use_model,
|
||||
"input": {"messages": [{"role": "user", "content": content_parts}]},
|
||||
"parameters": {"n": 1, "watermark": False, "prompt_extend": True}
|
||||
}
|
||||
headers = {"Content-Type": "application/json", "Authorization": f"Bearer {settings.AI_API_KEY}"}
|
||||
|
||||
log.info(f"{'='*60}")
|
||||
log.info(f"[ImageModel] DashScope 原生 API, 模型: {use_model}")
|
||||
log.info(f"[ImageModel] 图片数: {len(images_b64)}, 各图: {[len(b)//1024 for b in images_b64]}KB")
|
||||
image_inputs = [_image_data_url(img_b64) for img_b64 in images_b64] or None
|
||||
image_payload = None
|
||||
if image_inputs:
|
||||
image_payload = image_inputs[0] if len(image_inputs) == 1 else image_inputs
|
||||
log.info(f"{'=' * 60}")
|
||||
log.info(f"[ImageModel] provider={get_ai_provider(runtime_config)} model={use_model} source={(runtime_config.source if runtime_config else 'server')}")
|
||||
log.info(f"[ImageModel] 图片数: {len(images_b64)}")
|
||||
log.info(f"[ImageModel] 目标生成张数: {image_count}")
|
||||
log.info(f"[ImageModel] 提示词: {prompt[:200]}")
|
||||
|
||||
for idx, img_b64 in enumerate(images_b64):
|
||||
_save_debug_image(base64.b64decode(img_b64), f'input_{idx}.jpg')
|
||||
request_kwargs = {
|
||||
"model": use_model,
|
||||
"prompt": prompt,
|
||||
"image": image_payload,
|
||||
"response_format": "url",
|
||||
"size": size or "2K",
|
||||
"watermark": True,
|
||||
"stream": should_stream,
|
||||
"sequential_image_generation": "auto" if image_count > 1 else "disabled",
|
||||
}
|
||||
if image_payload is None:
|
||||
request_kwargs.pop("image")
|
||||
if image_count > 1:
|
||||
request_kwargs["sequential_image_generation_options"] = (
|
||||
SequentialImageGenerationOptions(max_images=image_count)
|
||||
)
|
||||
|
||||
resp = http_requests.post(api_url, json=payload, headers=headers, timeout=120)
|
||||
if resp.status_code != 200:
|
||||
error_data = resp.json() if resp.headers.get("content-type", "").startswith("application/json") else {}
|
||||
err_msg = error_data.get("message", resp.text[:300])
|
||||
log.error(f"[ImageModel] API 错误: {resp.status_code} {err_msg}")
|
||||
if "data_inspection_failed" in str(error_data):
|
||||
raise ValueError("图片内容未通过安全审核，请更换图片")
|
||||
raise ValueError(f"图片模型调用失败({resp.status_code}): {err_msg}")
|
||||
if should_stream:
|
||||
stream_response = client.images.generate(**request_kwargs)
|
||||
images: List[dict] = []
|
||||
usage: Optional[dict] = None
|
||||
errors: List[str] = []
|
||||
|
||||
data = resp.json()
|
||||
output = data.get("output", {})
|
||||
choices = output.get("choices", [])
|
||||
if not choices:
|
||||
raise ValueError("模型未返回结果")
|
||||
for event in stream_response:
|
||||
if event is None:
|
||||
continue
|
||||
|
||||
content_list = choices[0].get("message", {}).get("content", [])
|
||||
image_url = None
|
||||
description = ""
|
||||
for item in content_list:
|
||||
if isinstance(item, dict):
|
||||
if "image" in item:
|
||||
image_url = item["image"]
|
||||
elif "text" in item:
|
||||
description += item["text"]
|
||||
event_type = str(getattr(event, "type", "") or "")
|
||||
event_error = _image_response_error_message(event)
|
||||
|
||||
if not image_url:
|
||||
raise ValueError("模型未返回图片")
|
||||
if event_type.endswith("partial_failed"):
|
||||
if event_error:
|
||||
errors.append(event_error)
|
||||
continue
|
||||
|
||||
log.info(f"[ImageModel] 输出图片 URL: {image_url[:120]}...")
|
||||
try:
|
||||
out_resp = http_requests.get(image_url, timeout=60)
|
||||
if out_resp.status_code == 200:
|
||||
_save_debug_image(out_resp.content, 'output.png')
|
||||
except Exception:
|
||||
pass
|
||||
if event_type.endswith("partial_succeeded"):
|
||||
try:
|
||||
images.append(
|
||||
_image_item_to_result(
|
||||
event,
|
||||
index=int(getattr(event, "image_index", len(images))),
|
||||
)
|
||||
)
|
||||
except Exception as exc:
|
||||
errors.append(str(exc))
|
||||
continue
|
||||
|
||||
return image_url, description
|
||||
if event_type.endswith("completed"):
|
||||
usage = _image_usage_dict(event)
|
||||
|
||||
if errors and not images:
|
||||
raise ValueError("；".join(errors))
|
||||
if not images:
|
||||
raise ValueError("图片模型未返回结果")
|
||||
|
||||
images.sort(key=lambda item: int(item.get("index", 0)))
|
||||
return {"images": images, "usage": usage, "errors": errors}
|
||||
|
||||
response_kwargs = {
|
||||
"model": use_model,
|
||||
"prompt": prompt,
|
||||
"image": image_payload,
|
||||
"response_format": "url",
|
||||
"size": size or "2K",
|
||||
"stream": False,
|
||||
"watermark": True,
|
||||
"sequential_image_generation": "auto" if image_count > 1 else "disabled",
|
||||
"sequential_image_generation_options": (
|
||||
SequentialImageGenerationOptions(max_images=image_count)
|
||||
if image_count > 1
|
||||
else None
|
||||
),
|
||||
}
|
||||
if image_payload is None:
|
||||
response_kwargs.pop("image")
|
||||
|
||||
response = client.images.generate(**response_kwargs)
|
||||
|
||||
error_message = _image_response_error_message(response)
|
||||
if error_message:
|
||||
raise ValueError(error_message)
|
||||
|
||||
data = getattr(response, "data", None) or []
|
||||
if not data:
|
||||
raise ValueError("图片模型未返回结果")
|
||||
|
||||
images = [_image_item_to_result(item, index=idx) for idx, item in enumerate(data)]
|
||||
return {"images": images, "usage": _image_usage_dict(response), "errors": []}
|
||||
|
||||
|
||||
# ==================== 验证套图效果 ====================
|
||||
|
||||
def verify_pattern_result(garment_b64: str, canvas_b64: str, extra_prompt: str = None, vision_model: str = None) -> str:
|
||||
"""用视觉模型对比原始成衣和套图结果"""
|
||||
use_model = vision_model or settings.AI_VISION_MODEL
|
||||
|
||||
log.info(f"[Verify] 模型: {use_model}, 成衣: {len(garment_b64)//1024}KB, 画布: {len(canvas_b64)//1024}KB")
|
||||
|
||||
def verify_pattern_result(
|
||||
garment_b64: str,
|
||||
canvas_b64: str,
|
||||
extra_prompt: str = None,
|
||||
vision_model: str = None,
|
||||
runtime_config: Optional[RuntimeAiConfig] = None,
|
||||
) -> str:
|
||||
"""用当前视觉模型对比原始成衣和套图结果。"""
|
||||
verify_prompt = (
|
||||
"请对比这两张图片：第一张是原始成衣照片，第二张是套图结果。\n"
|
||||
"验证：1. 花样还原度 2. 裁片覆盖完整度 3. 对齐质量 4. 整体效果。给出评分(1-10)和改进建议。"
|
||||
"请对比这两张图片：第一张是原始成衣照片，第二张是套图结果。"
|
||||
"请从花样还原度、裁片覆盖完整度、对齐质量、整体效果四个方面评分，并给出改进建议。"
|
||||
)
|
||||
if extra_prompt:
|
||||
verify_prompt += f"\n用户补充：{extra_prompt}"
|
||||
|
||||
if is_gemini_model(use_model):
|
||||
return call_gemini(
|
||||
[{"role": "user", "content": "你是服装套图质量检验专家。"}, {"role": "user", "content": verify_prompt}],
|
||||
use_model, images_b64=[garment_b64, canvas_b64]
|
||||
)
|
||||
|
||||
from openai import OpenAI
|
||||
client = OpenAI(api_key=settings.AI_API_KEY, base_url=settings.AI_BASE_URL or "https://api.openai.com/v1")
|
||||
completion = client.chat.completions.create(
|
||||
model=use_model,
|
||||
messages=[
|
||||
{"role": "system", "content": "你是服装套图质量检验专家。"},
|
||||
{"role": "user", "content": [
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{garment_b64}"}},
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{canvas_b64}"}},
|
||||
{"type": "text", "text": verify_prompt},
|
||||
]},
|
||||
],
|
||||
return call_vision_messages(
|
||||
verify_prompt,
|
||||
[garment_b64, canvas_b64],
|
||||
model_override=vision_model or settings.AI_VISION_MODEL,
|
||||
runtime_config=runtime_config,
|
||||
)
|
||||
return completion.choices[0].message.content or ""
|
||||
|
||||
|
||||
# ==================== Mock ====================
|
||||
|
||||
|
||||
def mock_reply(message: str, has_image: bool = False) -> str:
|
||||
"""未配置 API Key 时的模拟回复"""
|
||||
"""未配置 API Key 时的模拟回复。"""
|
||||
if has_image:
|
||||
return "【模拟分析】收到图片。AI 分析功能需要配置 AI_API_KEY。"
|
||||
return "【模拟分析】收到图片。请先在 API 配置页填写你自己的转发 Key。"
|
||||
if "套图" in message:
|
||||
return "套图功能在「参数预设」页面。先选择花样组和裁片组，再添加规则，最后点击生成。"
|
||||
return "你好！我是 DesignerCEP AI 助手。你可以问我关于套图、裁片、对齐等功能的问题。"
|
||||
return "套图功能已就绪，但正式 AI 分析前请先在 API 配置页填写你自己的转发 Key。"
|
||||
return "你好！我是 DesignerCEP AI 助手。请先在 API 配置页填写你的转发地址和 Key，然后我再帮你执行聊天、看图和套图任务。"
|
||||
|
||||
|
||||
# ==================== 工具函数 ====================
|
||||
# ==================== 调试工具 ====================
|
||||
|
||||
|
||||
def _save_debug_image(data: bytes, filename: str):
|
||||
"""保存调试图片到 debug_images 目录"""
|
||||
import os, time
|
||||
debug_dir = os.path.join(os.path.dirname(__file__), '..', '..', '..', 'debug_images')
|
||||
"""保存调试图片到 debug_images 目录。"""
|
||||
import time
|
||||
|
||||
debug_dir = os.path.join(
|
||||
os.path.dirname(__file__), "..", "..", "..", "debug_images"
|
||||
)
|
||||
os.makedirs(debug_dir, exist_ok=True)
|
||||
ts = int(time.time())
|
||||
path = os.path.join(debug_dir, f'{ts}_{filename}')
|
||||
path = os.path.join(debug_dir, f"{ts}_{filename}")
|
||||
try:
|
||||
with open(path, 'wb') as f:
|
||||
f.write(data)
|
||||
with open(path, "wb") as file_obj:
|
||||
file_obj.write(data)
|
||||
log.info(f"[Debug] 图片已保存: {path}")
|
||||
except Exception as e:
|
||||
log.warning(f"[Debug] 保存失败: {e}")
|
||||
except Exception as exc:
|
||||
log.warning(f"[Debug] 保存失败: {exc}")
|
||||
|
||||
476
Server/app/api/v1/ai_pattern.py
Normal file
476
Server/app/api/v1/ai_pattern.py
Normal file
@@ -0,0 +1,476 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
AI 图案生成接口
|
||||
功能：生成预览图、面料图、裁切、细化
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Depends
|
||||
from pydantic import BaseModel
|
||||
from typing import Optional
|
||||
import json, base64, logging
|
||||
from app.core.config import settings
|
||||
from app.core.security import get_current_user
|
||||
from app.db import get_db
|
||||
from sqlalchemy.orm import Session
|
||||
from app.models.user import User
|
||||
from app.api.v1.ai_llm import (
|
||||
call_image_model,
|
||||
call_image_model_batch,
|
||||
get_runtime_ai_config,
|
||||
has_ai_config,
|
||||
)
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
if not log.handlers:
|
||||
log.setLevel(logging.INFO)
|
||||
log.propagate = False
|
||||
_h = logging.StreamHandler()
|
||||
_h.setFormatter(
|
||||
logging.Formatter("[%(asctime)s][%(name)s][%(levelname)s] %(message)s")
|
||||
)
|
||||
log.addHandler(_h)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# ==================== 数据模型 ====================
|
||||
|
||||
|
||||
class GeneratePatternRequest(BaseModel):
|
||||
image_base64: str
|
||||
canvas_base64: Optional[str] = None
|
||||
prompt: Optional[str] = None
|
||||
image_edit_model: Optional[str] = None
|
||||
|
||||
|
||||
class GenerateFabricRequest(BaseModel):
|
||||
image_base64: str
|
||||
fabric_info: dict
|
||||
image_edit_model: Optional[str] = None
|
||||
|
||||
|
||||
class GenerateDesignImagesRequest(BaseModel):
|
||||
prompt: str
|
||||
reference_image_base64: Optional[str] = None
|
||||
count: int = 1
|
||||
size: Optional[str] = "2K"
|
||||
image_edit_model: Optional[str] = None
|
||||
|
||||
|
||||
class CropPieceRequest(BaseModel):
|
||||
preview_base64: str
|
||||
canvas_width: int
|
||||
canvas_height: int
|
||||
piece_left: float
|
||||
piece_top: float
|
||||
piece_width: float
|
||||
piece_height: float
|
||||
piece_name: str
|
||||
padding: float = 0.15
|
||||
|
||||
|
||||
class RefinePieceRequest(BaseModel):
|
||||
cropped_base64: str
|
||||
piece_name: str
|
||||
pattern_type: str = "fill_pattern"
|
||||
aspect_ratio: Optional[str] = None
|
||||
piece_width: Optional[int] = None
|
||||
piece_height: Optional[int] = None
|
||||
prompt: Optional[str] = None
|
||||
image_edit_model: Optional[str] = None
|
||||
|
||||
|
||||
class ExtractPieceRequest(BaseModel):
|
||||
preview_base64: str
|
||||
piece_name: str
|
||||
piece_description: str = ""
|
||||
prompt: Optional[str] = None
|
||||
|
||||
|
||||
# ==================== 图案生成接口 ====================
|
||||
|
||||
|
||||
@router.post("/ai/generate-preview")
|
||||
async def generate_preview(
|
||||
data: GeneratePatternRequest,
|
||||
current_username: str = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""阶段1：成衣照片 + 裁片截图 → AI 生成带轮廓的花样预览图"""
|
||||
log.info(f"{'=' * 60}")
|
||||
log.info(
|
||||
f"[Preview] 收到预览请求, 成衣图: {len(data.image_base64) // 1024}KB, 裁片图: {len(data.canvas_base64) // 1024 if data.canvas_base64 else 0}KB"
|
||||
)
|
||||
if data.prompt:
|
||||
log.info(f"[Preview] 自定义提示词: {data.prompt[:150]}")
|
||||
user = db.query(User).filter(User.username == current_username).first()
|
||||
runtime_ai_config = get_runtime_ai_config(user)
|
||||
effective_image_model = (
|
||||
data.image_edit_model
|
||||
or (user.ai_image_model if user else None)
|
||||
or settings.AI_IMAGE_EDIT_MODEL
|
||||
)
|
||||
if not has_ai_config(runtime_ai_config) or not effective_image_model:
|
||||
raise HTTPException(400, "请先在 API 配置页填写转发 Key，或联系管理员配置默认 AI_API_KEY")
|
||||
|
||||
try:
|
||||
images = [data.image_base64]
|
||||
if data.canvas_base64:
|
||||
images.append(data.canvas_base64)
|
||||
|
||||
default_prompt = (
|
||||
(
|
||||
"Image 1 是一件成衣照片，Image 2 是裁片轮廓图。"
|
||||
"请将 Image 1 成衣上的面料花样提取出来，填充到 Image 2 的每个裁片轮廓内。"
|
||||
"要求：保持花样颜色、比例、方向一致；保留裁片轮廓线；不要改变裁片排列位置。"
|
||||
)
|
||||
if data.canvas_base64
|
||||
else ("请从这件成衣中提取面料花样，生成一张干净的花样平铺图。")
|
||||
)
|
||||
|
||||
result_url, desc = call_image_model(
|
||||
images_b64=images,
|
||||
prompt=data.prompt or default_prompt,
|
||||
model_override=effective_image_model,
|
||||
runtime_config=runtime_ai_config,
|
||||
)
|
||||
return {"code": 200, "data": {"image_url": result_url, "description": desc}}
|
||||
except Exception as e:
|
||||
log.error(f"预览生成失败: {e}", exc_info=True)
|
||||
err_msg = str(e)
|
||||
if "data_inspection_failed" in err_msg or "inappropriate" in err_msg:
|
||||
raise HTTPException(
|
||||
400,
|
||||
"图片内容未通过平台安全审核，请更换图片后重试（避免使用含版权角色/敏感内容的图片）",
|
||||
)
|
||||
raise HTTPException(500, f"预览生成失败: {err_msg}")
|
||||
|
||||
|
||||
@router.post("/ai/generate-fabric")
|
||||
async def generate_fabric(
|
||||
data: GenerateFabricRequest,
|
||||
current_username: str = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""all_over 模式：根据成衣照片 + fabric_info 生成面料平铺图"""
|
||||
log.info(f"{'=' * 60}")
|
||||
log.info(f"[Fabric] 生成面料图, 成衣: {len(data.image_base64) // 1024}KB")
|
||||
log.info(
|
||||
f"[Fabric] fabric_info: {json.dumps(data.fabric_info, ensure_ascii=False)}"
|
||||
)
|
||||
user = db.query(User).filter(User.username == current_username).first()
|
||||
runtime_ai_config = get_runtime_ai_config(user)
|
||||
effective_image_model = (
|
||||
data.image_edit_model
|
||||
or (user.ai_image_model if user else None)
|
||||
or settings.AI_IMAGE_EDIT_MODEL
|
||||
)
|
||||
if not has_ai_config(runtime_ai_config) or not effective_image_model:
|
||||
raise HTTPException(400, "请先在 API 配置页填写转发 Key，或联系管理员配置默认 AI_API_KEY")
|
||||
|
||||
try:
|
||||
fi = data.fabric_info
|
||||
prompt = (
|
||||
f"Please generate a high-resolution flat fabric pattern image based on this garment photo.\n"
|
||||
f"\nRequirements:\n"
|
||||
f"1. Output a RECTANGULAR flat fabric image (not garment shape, just the fabric laid flat)\n"
|
||||
f"2. Fabric type: {fi.get('fabric_type', 'digital print')}\n"
|
||||
f"3. Pattern elements: {fi.get('pattern_elements', 'floral')}\n"
|
||||
f"4. Pattern direction: {fi.get('pattern_direction', 'uniform')}\n"
|
||||
f"5. Color transition: {fi.get('color_transition', 'none')}\n"
|
||||
f"6. Pattern density: {fi.get('density', 'medium')}\n"
|
||||
f"7. Remove all garment wrinkles/shadows - show flat fabric only\n"
|
||||
f"8. The pattern must be clear, colors accurate, suitable for garment piece overlay\n"
|
||||
f"9. Aspect ratio: 3:4\n"
|
||||
f"10. The image should look like a piece of fabric photographed from directly above on a flat surface"
|
||||
)
|
||||
|
||||
log.info(f"[Fabric] 提示词: {prompt[:200]}")
|
||||
|
||||
result_url, desc = call_image_model(
|
||||
images_b64=[data.image_base64],
|
||||
prompt=prompt,
|
||||
model_override=effective_image_model,
|
||||
runtime_config=runtime_ai_config,
|
||||
)
|
||||
|
||||
log.info(f"[Fabric] 面料图生成完成")
|
||||
return {"code": 200, "data": {"fabric_url": result_url, "description": desc}}
|
||||
|
||||
except Exception as e:
|
||||
log.error(f"面料图生成失败: {e}", exc_info=True)
|
||||
raise HTTPException(500, f"面料图生成失败: {str(e)}")
|
||||
|
||||
|
||||
@router.post("/ai/generate-design-images")
|
||||
async def generate_design_images(
|
||||
data: GenerateDesignImagesRequest,
|
||||
current_username: str = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""通用豆包出图：支持纯文生图和基于参考图的连续组图。"""
|
||||
log.info(f"{'=' * 60}")
|
||||
log.info(
|
||||
f"[DesignImage] 通用出图, 参考图={'是' if data.reference_image_base64 else '否'}, count={data.count}, size={data.size or '2K'}"
|
||||
)
|
||||
user = db.query(User).filter(User.username == current_username).first()
|
||||
runtime_ai_config = get_runtime_ai_config(user)
|
||||
effective_image_model = (
|
||||
data.image_edit_model
|
||||
or (user.ai_image_model if user else None)
|
||||
or settings.AI_IMAGE_EDIT_MODEL
|
||||
)
|
||||
if not has_ai_config(runtime_ai_config) or not effective_image_model:
|
||||
raise HTTPException(400, "请先在 API 配置页填写转发 Key，或联系管理员配置默认 AI_API_KEY")
|
||||
|
||||
prompt = (data.prompt or "").strip()
|
||||
if not prompt:
|
||||
raise HTTPException(400, "prompt 不能为空")
|
||||
|
||||
try:
|
||||
count = max(1, min(int(data.count or 1), 4))
|
||||
images_b64 = [data.reference_image_base64] if data.reference_image_base64 else []
|
||||
result = call_image_model_batch(
|
||||
images_b64=images_b64,
|
||||
prompt=prompt,
|
||||
model_override=effective_image_model,
|
||||
size=data.size or "2K",
|
||||
max_images=count,
|
||||
stream=(count > 1),
|
||||
runtime_config=runtime_ai_config,
|
||||
)
|
||||
images = result.get("images", [])
|
||||
return {
|
||||
"code": 200,
|
||||
"data": {
|
||||
"images": images,
|
||||
"image_urls": [item["url"] for item in images],
|
||||
"count": len(images),
|
||||
"used_reference_image": bool(data.reference_image_base64),
|
||||
"usage": result.get("usage"),
|
||||
},
|
||||
}
|
||||
except Exception as e:
|
||||
log.error(f"通用出图失败: {e}", exc_info=True)
|
||||
err_msg = str(e)
|
||||
if "data_inspection_failed" in err_msg or "inappropriate" in err_msg:
|
||||
raise HTTPException(400, "图片内容未通过平台安全审核，请更换提示词或参考图")
|
||||
raise HTTPException(500, f"通用出图失败: {err_msg}")
|
||||
|
||||
|
||||
@router.post("/ai/crop-piece")
|
||||
async def crop_piece(
|
||||
data: CropPieceRequest, current_username: str = Depends(get_current_user)
|
||||
):
|
||||
"""从预览图中按坐标裁切指定裁片区域（Pillow，像素级精准）"""
|
||||
log.info(f"{'=' * 60}")
|
||||
log.info(f"[CropPiece] 裁切: {data.piece_name}")
|
||||
log.info(
|
||||
f"[CropPiece] 画布: {data.canvas_width}x{data.canvas_height}, 裁片区域: ({data.piece_left},{data.piece_top}) {data.piece_width}x{data.piece_height}"
|
||||
)
|
||||
|
||||
try:
|
||||
result_b64 = _crop_piece_from_preview(data)
|
||||
return {
|
||||
"code": 200,
|
||||
"data": {"cropped_base64": result_b64, "piece_name": data.piece_name},
|
||||
}
|
||||
except Exception as e:
|
||||
log.error(f"裁切失败: {e}", exc_info=True)
|
||||
raise HTTPException(500, f"裁切失败: {str(e)}")
|
||||
|
||||
|
||||
@router.post("/ai/refine-piece")
|
||||
async def refine_piece(
|
||||
data: RefinePieceRequest,
|
||||
current_username: str = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""AI 细化裁切图：根据图案类型生成不同提示词"""
|
||||
log.info(f"{'=' * 60}")
|
||||
log.info(
|
||||
f"[RefinePiece] 细化: {data.piece_name}, 类型: {data.pattern_type}, 比例: {data.aspect_ratio}"
|
||||
)
|
||||
log.info(
|
||||
f"[RefinePiece] 裁片尺寸: {data.piece_width}x{data.piece_height}, 图片: {len(data.cropped_base64) // 1024}KB"
|
||||
)
|
||||
|
||||
user = db.query(User).filter(User.username == current_username).first()
|
||||
runtime_ai_config = get_runtime_ai_config(user)
|
||||
effective_image_model = (
|
||||
data.image_edit_model
|
||||
or (user.ai_image_model if user else None)
|
||||
or settings.AI_IMAGE_EDIT_MODEL
|
||||
)
|
||||
if not has_ai_config(runtime_ai_config) or not effective_image_model:
|
||||
raise HTTPException(400, "请先在 API 配置页填写转发 Key，或联系管理员配置默认 AI_API_KEY")
|
||||
|
||||
try:
|
||||
size_hint = ""
|
||||
if data.aspect_ratio:
|
||||
size_hint = f"输出图片宽高比必须为 {data.aspect_ratio}。"
|
||||
if data.piece_width and data.piece_height:
|
||||
size_hint += f"输出尺寸至少 {int(data.piece_width * 1.15)}x{int(data.piece_height * 1.15)} 像素。"
|
||||
|
||||
if data.pattern_type == "fill_pattern":
|
||||
prompt = data.prompt or (
|
||||
f"这是从服装裁片中裁切出来的花型图案区域。"
|
||||
f"请将它处理为一张干净的矩形花型图："
|
||||
f"去掉所有轮廓线、标签文字和边缘空白；"
|
||||
f"花型图案要完整铺满整个矩形，无空白边距；"
|
||||
f"向四周自然延展重复纹样，保持花型连续无接缝；"
|
||||
f"保持花样的颜色和细节清晰。{size_hint}"
|
||||
)
|
||||
elif data.pattern_type == "theme_pattern":
|
||||
prompt = data.prompt or (
|
||||
f"这是从服装裁片中裁切出来的主题图案区域（如卡通人物、Logo、印花）。"
|
||||
f"请提取其中的主题图案，输出为纯白色背景(#FFFFFF)上的主题图案："
|
||||
f"主题图案要完整清晰，保持原始颜色和细节；"
|
||||
f"背景必须是纯白色(#FFFFFF)，不要任何其他底色或纹理；"
|
||||
f"去掉轮廓线和标签文字；主题图案在画面中居中放置。{size_hint}"
|
||||
)
|
||||
elif data.pattern_type == "mixed_fill":
|
||||
prompt = data.prompt or (
|
||||
f"这是从服装裁片中裁切出来的区域，包含主题图案和花型底纹。"
|
||||
f"请只提取底部的花型/纹理图案，去掉上面的主题图案（人物/Logo）："
|
||||
f"用底纹自然填充主题图案原来的位置；"
|
||||
f"花型要完整铺满整个矩形，保持纹样连续；"
|
||||
f"去掉轮廓线和标签文字。{size_hint}"
|
||||
)
|
||||
elif data.pattern_type == "mixed_theme":
|
||||
prompt = data.prompt or (
|
||||
f"这是从服装裁片中裁切出来的区域，包含主题图案和花型底纹。"
|
||||
f"请只提取主题图案（人物/Logo/大面积印花），输出为纯白色背景(#FFFFFF)："
|
||||
f"主题图案要完整清晰，保持原始颜色和细节；"
|
||||
f"背景必须是纯白色(#FFFFFF)，去掉所有底纹；"
|
||||
f"去掉轮廓线和标签文字；主题图案在画面中居中放置。{size_hint}"
|
||||
)
|
||||
else:
|
||||
prompt = (
|
||||
data.prompt
|
||||
or f"细化这张图片，去掉轮廓线和标签文字，保持清晰。{size_hint}"
|
||||
)
|
||||
|
||||
log.info(f"[RefinePiece] 提示词: {prompt[:200]}")
|
||||
|
||||
result_url, desc = call_image_model(
|
||||
images_b64=[data.cropped_base64],
|
||||
prompt=prompt,
|
||||
model_override=effective_image_model,
|
||||
runtime_config=runtime_ai_config,
|
||||
)
|
||||
return {
|
||||
"code": 200,
|
||||
"data": {
|
||||
"refined_url": result_url,
|
||||
"piece_name": data.piece_name,
|
||||
"pattern_type": data.pattern_type,
|
||||
},
|
||||
}
|
||||
except Exception as e:
|
||||
log.error(f"细化失败: {e}", exc_info=True)
|
||||
err_msg = str(e)
|
||||
if "data_inspection_failed" in err_msg:
|
||||
raise HTTPException(400, "图片内容未通过安全审核")
|
||||
raise HTTPException(500, f"细化失败: {err_msg}")
|
||||
|
||||
|
||||
@router.post("/ai/extract-piece-pattern")
|
||||
async def extract_piece_pattern(
|
||||
data: ExtractPieceRequest,
|
||||
current_username: str = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""阶段2：从预览图中提取指定裁片区域 → 输出矩形花样图"""
|
||||
log.info(f"{'=' * 60}")
|
||||
log.info(
|
||||
f"[ExtractPiece] 裁片: {data.piece_name}, 描述: '{data.piece_description}', 预览图: {len(data.preview_base64) // 1024}KB"
|
||||
)
|
||||
user = db.query(User).filter(User.username == current_username).first()
|
||||
runtime_ai_config = get_runtime_ai_config(user)
|
||||
effective_image_model = (
|
||||
getattr(data, "image_edit_model", None)
|
||||
or (user.ai_image_model if user else None)
|
||||
or settings.AI_IMAGE_EDIT_MODEL
|
||||
)
|
||||
if not has_ai_config(runtime_ai_config) or not effective_image_model:
|
||||
raise HTTPException(400, "请先在 API 配置页填写转发 Key，或联系管理员配置默认 AI_API_KEY")
|
||||
|
||||
try:
|
||||
prompt = data.prompt or (
|
||||
f"这是一张服装裁片花样预览图，包含多个裁片。"
|
||||
f"请提取其中【{data.piece_name}】的花样区域"
|
||||
f"{'（' + data.piece_description + '）' if data.piece_description else ''}，"
|
||||
f"将该区域的花样输出为一张完整的矩形图片。"
|
||||
f"要求：只保留花样内容，去掉轮廓线，填满整个矩形，保持清晰。"
|
||||
)
|
||||
result_url, desc = call_image_model(
|
||||
images_b64=[data.preview_base64],
|
||||
prompt=prompt,
|
||||
model_override=effective_image_model,
|
||||
runtime_config=runtime_ai_config,
|
||||
)
|
||||
return {
|
||||
"code": 200,
|
||||
"data": {
|
||||
"pattern_url": result_url,
|
||||
"piece_name": data.piece_name,
|
||||
"description": desc,
|
||||
},
|
||||
}
|
||||
except Exception as e:
|
||||
log.error(f"裁片提取失败: {e}", exc_info=True)
|
||||
raise HTTPException(500, f"裁片提取失败: {str(e)}")
|
||||
|
||||
|
||||
# ==================== 工具函数 ====================
|
||||
|
||||
|
||||
def _crop_piece_from_preview(data: CropPieceRequest) -> str:
|
||||
"""用 Pillow 从预览图中按 PS 坐标裁切指定区域（带出血线）"""
|
||||
from PIL import Image
|
||||
import io
|
||||
|
||||
img_bytes = base64.b64decode(data.preview_base64)
|
||||
preview_img = Image.open(io.BytesIO(img_bytes))
|
||||
pw, ph = preview_img.size
|
||||
|
||||
scale_x = pw / data.canvas_width
|
||||
scale_y = ph / data.canvas_height
|
||||
|
||||
bleed_x = data.piece_width * data.padding
|
||||
bleed_y = data.piece_height * data.padding
|
||||
|
||||
left = max(0, (data.piece_left - bleed_x) * scale_x)
|
||||
top = max(0, (data.piece_top - bleed_y) * scale_y)
|
||||
right = min(pw, (data.piece_left + data.piece_width + bleed_x) * scale_x)
|
||||
bottom = min(ph, (data.piece_top + data.piece_height + bleed_y) * scale_y)
|
||||
|
||||
log.info(f"[CropPiece] PS 画布: {data.canvas_width}x{data.canvas_height}")
|
||||
log.info(f"[CropPiece] 预览图: {pw}x{ph}")
|
||||
log.info(f"[CropPiece] 缩放比: x={scale_x:.4f}, y={scale_y:.4f}")
|
||||
log.info(
|
||||
f"[CropPiece] PS 裁片区域: ({data.piece_left},{data.piece_top}) {data.piece_width}x{data.piece_height}"
|
||||
)
|
||||
log.info(
|
||||
f"[CropPiece] 出血线: {data.padding * 100:.0f}% → x±{bleed_x:.0f}px, y±{bleed_y:.0f}px"
|
||||
)
|
||||
log.info(
|
||||
f"[CropPiece] 预览图裁切: ({left:.0f},{top:.0f})-({right:.0f},{bottom:.0f}) = {right - left:.0f}x{bottom - top:.0f}px"
|
||||
)
|
||||
|
||||
cropped = preview_img.crop((int(left), int(top), int(right), int(bottom)))
|
||||
|
||||
import os, time
|
||||
|
||||
debug_dir = os.path.join(
|
||||
os.path.dirname(__file__), "..", "..", "..", "debug_images"
|
||||
)
|
||||
os.makedirs(debug_dir, exist_ok=True)
|
||||
ts = int(time.time())
|
||||
debug_path = os.path.join(debug_dir, f"{ts}_crop_{data.piece_name}.png")
|
||||
cropped.save(debug_path)
|
||||
log.info(f"[CropPiece] 裁切结果已保存: {debug_path}")
|
||||
|
||||
buf = io.BytesIO()
|
||||
cropped.save(buf, format="PNG")
|
||||
return base64.b64encode(buf.getvalue()).decode()
|
||||
636
Server/app/api/v1/ai_skills.py
Normal file
636
Server/app/api/v1/ai_skills.py
Normal file
@@ -0,0 +1,636 @@
|
||||
"""
|
||||
AI 技能注册表
|
||||
为 AI 助手提供可复用的任务模式、工作流和工具边界。
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Iterable, Optional
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AiSkill:
|
||||
id: str
|
||||
name: str
|
||||
description: str
|
||||
mode: str
|
||||
keywords: tuple[str, ...]
|
||||
allowed_tools: tuple[str, ...]
|
||||
workflow: tuple[str, ...]
|
||||
guardrails: tuple[str, ...]
|
||||
success_criteria: tuple[str, ...]
|
||||
planning_style: str
|
||||
image_hint: str = ""
|
||||
origin: str = ""
|
||||
origin_url: str = ""
|
||||
upstream_skill: str = ""
|
||||
triage_questions: tuple[str, ...] = ()
|
||||
deliverables: tuple[str, ...] = ()
|
||||
execution_notes: tuple[str, ...] = ()
|
||||
|
||||
def to_public_dict(self) -> dict:
|
||||
return {
|
||||
"id": self.id,
|
||||
"name": self.name,
|
||||
"description": self.description,
|
||||
"mode": self.mode,
|
||||
"planning_style": self.planning_style,
|
||||
"image_hint": self.image_hint,
|
||||
"origin": self.origin,
|
||||
"origin_url": self.origin_url,
|
||||
"upstream_skill": self.upstream_skill,
|
||||
}
|
||||
|
||||
|
||||
AUTO_SKILL = {
|
||||
"id": "auto",
|
||||
"name": "智能自动选择",
|
||||
"description": "按当前问题、图片和 Photoshop 上下文，自动切换最合适的技能。",
|
||||
"mode": "auto",
|
||||
"planning_style": "先判断任务类型，再切到最匹配的专家技能。",
|
||||
"image_hint": "适合不知道该选哪个技能时使用。",
|
||||
"origin": "内置技能路由",
|
||||
"origin_url": "",
|
||||
"upstream_skill": "",
|
||||
}
|
||||
|
||||
OPENCLAW_REPO_URL = "https://github.com/openclaw/skills"
|
||||
OPENCLAW_PS_AUTOMATOR_URL = (
|
||||
"https://github.com/openclaw/skills/tree/main/skills/abdul-karim-mia/photoshop-automator"
|
||||
)
|
||||
OPENCLAW_UI_UX_PRO_URL = (
|
||||
"https://github.com/openclaw/skills/tree/main/skills/15349185792/ui-ux-pro-max-0-1-0"
|
||||
)
|
||||
OPENCLAW_UI_DESIGNER_URL = (
|
||||
"https://github.com/openclaw/skills/tree/main/skills/1999azzar/ui-designer-skill"
|
||||
)
|
||||
|
||||
|
||||
SKILLS: tuple[AiSkill, ...] = (
|
||||
AiSkill(
|
||||
id="ps-generalist",
|
||||
name="PS 全能助手",
|
||||
description="处理一般 Photoshop 操作、图层整理、文档检查、稳定执行和轻量自动化任务。",
|
||||
mode="tool_agent",
|
||||
keywords=(
|
||||
"ps",
|
||||
"photoshop",
|
||||
"图层",
|
||||
"文档",
|
||||
"画布",
|
||||
"对齐",
|
||||
"移动",
|
||||
"旋转",
|
||||
"缩放",
|
||||
"文字",
|
||||
"保存",
|
||||
"导出",
|
||||
"出图",
|
||||
"效果图",
|
||||
"不会",
|
||||
"怎么做",
|
||||
"怎么操作",
|
||||
"如何操作",
|
||||
"教我",
|
||||
"报错",
|
||||
"错误",
|
||||
"失败",
|
||||
"没反应",
|
||||
"不能",
|
||||
"无法",
|
||||
"为什么",
|
||||
),
|
||||
allowed_tools=(
|
||||
"get_document_info",
|
||||
"get_layer_structure",
|
||||
"get_active_layer_info",
|
||||
"list_all_layer_names",
|
||||
"list_documents",
|
||||
"switch_document",
|
||||
"create_layer",
|
||||
"rename_layer",
|
||||
"delete_layer",
|
||||
"duplicate_layer",
|
||||
"create_layer_group",
|
||||
"move_layer_to_group",
|
||||
"group_layers",
|
||||
"move_layer",
|
||||
"resize_layer",
|
||||
"rotate_layer",
|
||||
"flip_layer",
|
||||
"align_layers",
|
||||
"align_layers_tool",
|
||||
"distribute_layers",
|
||||
"set_layer_visible",
|
||||
"set_layer_opacity",
|
||||
"set_layer_blend_mode",
|
||||
"bring_to_front",
|
||||
"send_to_back",
|
||||
"resize_canvas",
|
||||
"create_text_layer",
|
||||
"set_text_content",
|
||||
"set_text_color",
|
||||
"set_text_size",
|
||||
"generate_design_images",
|
||||
"save_document",
|
||||
"save_document_as",
|
||||
"undo",
|
||||
"close_document",
|
||||
),
|
||||
workflow=(
|
||||
"先确认当前活动文档、目标图层和任务边界，再决定操作路径。",
|
||||
"优先用最少的步骤完成任务，不做多余改动，能查就先查。",
|
||||
"执行后总结结果，并明确告诉用户下一步还能继续做什么。",
|
||||
),
|
||||
guardrails=(
|
||||
"涉及删除、关闭、覆盖保存时先征求确认。",
|
||||
"不清楚目标图层时先查询，不要盲改。",
|
||||
"能在新图层或新图层组完成的改动，不直接破坏原图层。",
|
||||
"Photoshop 若有模态弹窗或保存窗口，先提醒用户关闭，再继续工具执行。",
|
||||
"按图层名操作时要求精确匹配，不要自己猜测最像的图层。",
|
||||
),
|
||||
success_criteria=(
|
||||
"操作路径清晰",
|
||||
"图层关系不混乱",
|
||||
"文档仍然可继续编辑",
|
||||
),
|
||||
planning_style="先查活动文档和目标图层，再决定是查询、轻改还是执行一串可回退的 PS 操作。",
|
||||
image_hint="适合边聊边操作 Photoshop，而不是深度看图分析。",
|
||||
origin="改造自 OpenClaw 的 photoshop-automator",
|
||||
origin_url=OPENCLAW_PS_AUTOMATOR_URL,
|
||||
upstream_skill="openclaw/photoshop-automator",
|
||||
execution_notes=(
|
||||
"默认围绕当前活动文档工作；没有文档时先提示用户打开文件。",
|
||||
"执行链路尽量短、原子化，避免一次堆很多风险操作。",
|
||||
"遇到文本替换、图层修改这类任务时，先确认名称和对象，再执行。",
|
||||
"如果用户是在问不会怎么做、为什么失败或下一步怎么操作，先读取当前上下文再答疑，必要时同轮直接执行。",
|
||||
),
|
||||
),
|
||||
AiSkill(
|
||||
id="garment-pattern-mapper",
|
||||
name="服装套图师",
|
||||
description="处理服装成衣、裁片、花型、面料和平铺套图任务。",
|
||||
mode="tool_agent",
|
||||
keywords=(
|
||||
"套图",
|
||||
"裁片",
|
||||
"成衣",
|
||||
"服装",
|
||||
"花型",
|
||||
"花样",
|
||||
"印花",
|
||||
"面料",
|
||||
"前片",
|
||||
"后片",
|
||||
"袖子",
|
||||
"pattern",
|
||||
"garment",
|
||||
),
|
||||
allowed_tools=(
|
||||
"get_document_info",
|
||||
"get_layer_structure",
|
||||
"list_all_layer_names",
|
||||
"identify_pieces",
|
||||
"generate_garment_preview",
|
||||
"extract_and_apply_all_pieces",
|
||||
"verify_pattern_result",
|
||||
),
|
||||
workflow=(
|
||||
"先判断用户是要识别、预览还是正式套图。",
|
||||
"需要套图时，先识别裁片和花型模式，再生成预览或直接走 all_over 流程。",
|
||||
"正式套图后用验证结果总结问题和改进建议。",
|
||||
),
|
||||
guardrails=(
|
||||
"生成预览后，除非用户明确确认，否则不要直接进入正式套图。",
|
||||
"识别结果不确定时要先说明，再继续下一步。",
|
||||
"没有参考图或没有有效裁片上下文时，不要假装已经完成套图。",
|
||||
),
|
||||
success_criteria=(
|
||||
"裁片识别合理",
|
||||
"花型方向和比例尽量接近原图",
|
||||
"流程状态对用户透明",
|
||||
),
|
||||
planning_style="把任务拆成 识别 -> 预览 -> 确认 -> 正式套图 -> 验证 五段式流程。",
|
||||
image_hint="适合用户上传成衣图、要求识别裁片、生成预览或正式套图时使用。",
|
||||
),
|
||||
AiSkill(
|
||||
id="ps-layout-designer",
|
||||
name="PS 排版设计师",
|
||||
description="处理海报、详情页、主视觉、标题层级、品牌感和版式整理任务。",
|
||||
mode="tool_agent",
|
||||
keywords=(
|
||||
"海报",
|
||||
"排版",
|
||||
"版式",
|
||||
"标题",
|
||||
"字体",
|
||||
"封面",
|
||||
"主视觉",
|
||||
"详情页",
|
||||
"banner",
|
||||
"构图",
|
||||
"留白",
|
||||
"层级",
|
||||
"品牌感",
|
||||
"视觉方向",
|
||||
"卖点",
|
||||
"背景图",
|
||||
"效果图",
|
||||
"方案图",
|
||||
),
|
||||
allowed_tools=(
|
||||
"get_document_info",
|
||||
"get_layer_structure",
|
||||
"list_all_layer_names",
|
||||
"get_active_layer_info",
|
||||
"create_text_layer",
|
||||
"set_text_content",
|
||||
"set_text_color",
|
||||
"set_text_size",
|
||||
"create_layer_group",
|
||||
"group_layers",
|
||||
"move_layer",
|
||||
"resize_layer",
|
||||
"align_layers_tool",
|
||||
"distribute_layers",
|
||||
"bring_to_front",
|
||||
"send_to_back",
|
||||
"set_layer_opacity",
|
||||
"set_layer_blend_mode",
|
||||
"add_stroke",
|
||||
"add_drop_shadow",
|
||||
"generate_design_images",
|
||||
),
|
||||
workflow=(
|
||||
"先分析画布比例、已有图层、信息密度和视觉重心。",
|
||||
"先给出版式思路，再搭主标题、辅文、卖点和装饰层。",
|
||||
"最后统一对齐、层级、间距、可读性和品牌感。",
|
||||
),
|
||||
guardrails=(
|
||||
"涉及大改版式时，优先新建组或新图层，而不是覆盖原设计。",
|
||||
"没有足够上下文时，先做查询或提出一版轻量方案。",
|
||||
"不要一次性做太多不可逆操作。",
|
||||
),
|
||||
success_criteria=(
|
||||
"视觉层级清楚",
|
||||
"对齐和间距统一",
|
||||
"画面有主次和留白",
|
||||
),
|
||||
planning_style="先做版式诊断和信息架构，再分步搭建图层，不要一上来就乱动。",
|
||||
image_hint="适合海报排版、标题优化、版式调整和页面重构。",
|
||||
origin="改造自 OpenClaw 的 ui-ux-pro-max / ui-designer-skill",
|
||||
origin_url=OPENCLAW_UI_UX_PRO_URL,
|
||||
upstream_skill="openclaw/ui-ux-pro-max + ui-designer-skill",
|
||||
triage_questions=(
|
||||
"当前作品更偏海报、封面、电商主图还是详情页？",
|
||||
"这次要优先提升点击率、品牌感、层级清晰度还是信息转化？",
|
||||
"有没有必须保留的品牌色、主标题、卖点或参考风格？",
|
||||
),
|
||||
deliverables=(
|
||||
"一句话视觉方向",
|
||||
"标题/副标题/卖点的层级方案",
|
||||
"对齐、留白、构图和配色建议",
|
||||
"对应到 Photoshop 的图层调整动作",
|
||||
),
|
||||
execution_notes=(
|
||||
"先拆信息优先级，再决定字体大小、位置和装饰强度。",
|
||||
"尽量在新组内搭结构，保留原稿便于回退和对比。",
|
||||
"如果上下文不足，先给一版轻量方向，再等用户确认后深化。",
|
||||
),
|
||||
),
|
||||
AiSkill(
|
||||
id="creative-direction-strategist",
|
||||
name="创意方向设计师",
|
||||
description="处理参考图拆解、品牌调性、视觉方向、配色与构图方案，把灵感转成可执行设计路线。",
|
||||
mode="hybrid_chat",
|
||||
keywords=(
|
||||
"创意",
|
||||
"方向",
|
||||
"灵感",
|
||||
"调性",
|
||||
"参考图",
|
||||
"品牌感",
|
||||
"构图",
|
||||
"配色",
|
||||
"氛围",
|
||||
"风格方案",
|
||||
"视觉方向",
|
||||
"art direction",
|
||||
"moodboard",
|
||||
"方向图",
|
||||
"概念图",
|
||||
"效果图",
|
||||
"出图",
|
||||
"方案图",
|
||||
"插画",
|
||||
"画面",
|
||||
"方向稿",
|
||||
"素材图",
|
||||
),
|
||||
allowed_tools=(
|
||||
"get_document_info",
|
||||
"get_layer_structure",
|
||||
"list_all_layer_names",
|
||||
"create_layer_group",
|
||||
"create_text_layer",
|
||||
"align_layers_tool",
|
||||
"add_guide",
|
||||
"generate_design_images",
|
||||
),
|
||||
workflow=(
|
||||
"先理解任务目标和参考图里最有价值的视觉线索。",
|
||||
"再给出 2 到 3 条创意方向，说明各自的调性、构图和适用场景。",
|
||||
"最后把选中的方向转成 Photoshop 可执行的排版、图层和颜色建议。",
|
||||
),
|
||||
guardrails=(
|
||||
"不要只给抽象评价，要把风格结论落到可执行动作上。",
|
||||
"没有足够上下文时，不要假设品牌调性和目标人群。",
|
||||
"除非用户要求，不直接大改现有图层结构。",
|
||||
),
|
||||
success_criteria=(
|
||||
"风格判断可信",
|
||||
"方向方案有区分度",
|
||||
"建议能直接落回 Photoshop",
|
||||
),
|
||||
planning_style="先做视觉诊断，再给 2 到 3 条方向，再落成可执行的排版和图层建议。",
|
||||
image_hint="适合上传参考图后，让 AI 像设计师一样拆方向、配色、构图和执行路径。",
|
||||
origin="改造自 OpenClaw 的 ui-ux-pro-max / ui-designer-skill",
|
||||
origin_url=OPENCLAW_REPO_URL,
|
||||
upstream_skill="openclaw/ui-ux-pro-max + ui-designer-skill",
|
||||
triage_questions=(
|
||||
"这次更偏品牌升级、活动海报、电商转化还是氛围图？",
|
||||
"想保留参考图里的哪些东西：配色、构图、质感还是情绪？",
|
||||
"输出要更稳重、高级、年轻还是更强转化？",
|
||||
),
|
||||
deliverables=(
|
||||
"2 到 3 条创意方向",
|
||||
"主视觉、标题、辅助元素的布局思路",
|
||||
"配色、字体和材质建议",
|
||||
"进入 Photoshop 后的第一轮动作清单",
|
||||
),
|
||||
execution_notes=(
|
||||
"有参考图时先拆视觉语言，再给风格方向，不要直接开始操作。",
|
||||
"没有图片时也要先问清目标和场景，再给方案。",
|
||||
"当用户确认方向后，再切到排版或修图类技能深入执行。",
|
||||
),
|
||||
),
|
||||
AiSkill(
|
||||
id="product-retoucher",
|
||||
name="修图精修师",
|
||||
description="处理抠图、调色、去背、产品主图优化、亮度对比和图层修整任务。",
|
||||
mode="tool_agent",
|
||||
keywords=(
|
||||
"修图",
|
||||
"精修",
|
||||
"抠图",
|
||||
"去背",
|
||||
"调色",
|
||||
"亮度",
|
||||
"对比度",
|
||||
"磨皮",
|
||||
"主图",
|
||||
"高光",
|
||||
"阴影",
|
||||
"retouch",
|
||||
),
|
||||
allowed_tools=(
|
||||
"get_document_info",
|
||||
"get_layer_structure",
|
||||
"get_active_layer_info",
|
||||
"duplicate_layer",
|
||||
"create_layer",
|
||||
"rasterize_layer",
|
||||
"adjust_brightness_contrast",
|
||||
"adjust_levels",
|
||||
"adjust_hsl",
|
||||
"auto_levels",
|
||||
"auto_contrast",
|
||||
"desaturate",
|
||||
"gaussian_blur",
|
||||
"create_clipping_mask",
|
||||
"release_clipping_mask",
|
||||
"add_layer_mask",
|
||||
"delete_layer_mask",
|
||||
"fill_selection",
|
||||
"inverse_selection",
|
||||
"feather_selection",
|
||||
"copy_merged_to_new_layer",
|
||||
"set_layer_blend_mode",
|
||||
"set_layer_opacity",
|
||||
"save_document",
|
||||
"save_document_as",
|
||||
),
|
||||
workflow=(
|
||||
"先判断任务属于结构修图、颜色修正还是产品精修。",
|
||||
"优先做可回退的副本或蒙版，再进行调整。",
|
||||
"最终说明改动方向和还可以继续优化的点。",
|
||||
),
|
||||
guardrails=(
|
||||
"不要直接破坏唯一原图层。",
|
||||
"没有明确选区或图层时先查询上下文。",
|
||||
"涉及批量覆盖时先提醒用户风险。",
|
||||
),
|
||||
success_criteria=(
|
||||
"改动可回退",
|
||||
"主体更清晰",
|
||||
"颜色和层次更稳定",
|
||||
),
|
||||
planning_style="优先非破坏性修图：复制、蒙版、调整，再视情况合并。",
|
||||
image_hint="适合产品主图、商品精修、调色和去背类任务。",
|
||||
),
|
||||
AiSkill(
|
||||
id="visual-analysis-advisor",
|
||||
name="看图分析师",
|
||||
description="处理上传图片后的版型、风格、配色、元素拆解、设计建议和参考分析。",
|
||||
mode="vision_chat",
|
||||
keywords=(
|
||||
"看图",
|
||||
"分析",
|
||||
"你看见",
|
||||
"这张图",
|
||||
"参考图",
|
||||
"风格",
|
||||
"版型",
|
||||
"配色",
|
||||
"元素",
|
||||
"灵感",
|
||||
"建议",
|
||||
"评价",
|
||||
),
|
||||
allowed_tools=(),
|
||||
workflow=(
|
||||
"先解释看到了什么，再拆解风格、结构和重点元素。",
|
||||
"如果用户有执行目标，再把分析转成可落地的 Photoshop 操作建议。",
|
||||
),
|
||||
guardrails=(
|
||||
"不要虚构已经执行过 Photoshop 操作。",
|
||||
"分析不确定时用概率表达，不要装作绝对正确。",
|
||||
),
|
||||
success_criteria=(
|
||||
"描述准确",
|
||||
"建议可执行",
|
||||
"重点突出",
|
||||
),
|
||||
planning_style="先看图，再给结论，最后给下一步操作建议。",
|
||||
image_hint="适合用户上传图片后问“你看见了什么”“这个版型/风格怎么做”。",
|
||||
origin="改造自 OpenClaw 的 ui-designer-skill",
|
||||
origin_url=OPENCLAW_UI_DESIGNER_URL,
|
||||
upstream_skill="openclaw/ui-designer-skill",
|
||||
deliverables=(
|
||||
"图片内容描述",
|
||||
"风格、构图、配色和元素拆解",
|
||||
"可以继续在 Photoshop 落地的建议",
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
SKILL_BY_ID = {skill.id: skill for skill in SKILLS}
|
||||
|
||||
|
||||
def list_ai_skills_public() -> list[dict]:
|
||||
return [AUTO_SKILL, *(skill.to_public_dict() for skill in SKILLS)]
|
||||
|
||||
|
||||
def get_ai_skill(skill_id: Optional[str]) -> Optional[AiSkill]:
|
||||
if not skill_id or skill_id == "auto":
|
||||
return None
|
||||
return SKILL_BY_ID.get(skill_id)
|
||||
|
||||
|
||||
def _history_text(history_messages: Iterable[dict]) -> str:
|
||||
parts: list[str] = []
|
||||
for item in history_messages:
|
||||
content = str(item.get("content", "")).strip()
|
||||
if content:
|
||||
parts.append(content)
|
||||
return "\n".join(parts[-6:])
|
||||
|
||||
|
||||
def _score_skill(skill: AiSkill, text: str, has_image: bool) -> int:
|
||||
lowered = text.lower()
|
||||
score = 0
|
||||
|
||||
for keyword in skill.keywords:
|
||||
if keyword.lower() in lowered:
|
||||
score += 3
|
||||
|
||||
if has_image and skill.mode == "vision_chat":
|
||||
score += 2
|
||||
if has_image and skill.id == "garment-pattern-mapper":
|
||||
score += 1
|
||||
|
||||
if skill.id == "garment-pattern-mapper" and any(
|
||||
token in lowered for token in ("套图", "裁片", "成衣", "花型", "面料", "pattern")
|
||||
):
|
||||
score += 6
|
||||
if skill.id == "ps-layout-designer" and any(
|
||||
token in lowered
|
||||
for token in (
|
||||
"排版",
|
||||
"海报",
|
||||
"标题",
|
||||
"版式",
|
||||
"封面",
|
||||
"banner",
|
||||
"品牌感",
|
||||
"卖点",
|
||||
"背景图",
|
||||
"方案图",
|
||||
)
|
||||
):
|
||||
score += 5
|
||||
if skill.id == "creative-direction-strategist" and any(
|
||||
token in lowered
|
||||
for token in (
|
||||
"创意",
|
||||
"方向",
|
||||
"灵感",
|
||||
"调性",
|
||||
"参考图",
|
||||
"品牌感",
|
||||
"构图",
|
||||
"配色",
|
||||
"氛围",
|
||||
"风格方案",
|
||||
"方向图",
|
||||
"效果图",
|
||||
"概念图",
|
||||
"方案图",
|
||||
"背景图",
|
||||
"出图",
|
||||
"生成一张",
|
||||
"生成四张",
|
||||
"生成一组",
|
||||
"插画",
|
||||
"画面",
|
||||
"方向稿",
|
||||
"连贯",
|
||||
)
|
||||
):
|
||||
score += 6
|
||||
if has_image and skill.id == "creative-direction-strategist":
|
||||
score += 2
|
||||
if skill.id == "product-retoucher" and any(
|
||||
token in lowered for token in ("修图", "去背", "抠图", "调色", "精修", "主图")
|
||||
):
|
||||
score += 5
|
||||
if skill.id == "visual-analysis-advisor" and has_image and any(
|
||||
token in lowered for token in ("分析", "看", "风格", "版型", "你看见", "建议", "元素")
|
||||
):
|
||||
score += 6
|
||||
if skill.id == "ps-generalist" and any(
|
||||
token in lowered
|
||||
for token in (
|
||||
"不会",
|
||||
"怎么做",
|
||||
"怎么操作",
|
||||
"如何操作",
|
||||
"教我",
|
||||
"帮我操作",
|
||||
"帮我弄",
|
||||
"为什么",
|
||||
"报错",
|
||||
"错误",
|
||||
"失败",
|
||||
"没反应",
|
||||
"不能",
|
||||
"无法",
|
||||
"下一步",
|
||||
"该怎么",
|
||||
)
|
||||
):
|
||||
score += 7
|
||||
if skill.mode == "tool_agent" and any(
|
||||
token in lowered for token in ("帮我做", "直接做", "你来做", "顺手做", "帮我操作")
|
||||
):
|
||||
score += 4
|
||||
|
||||
return score
|
||||
|
||||
|
||||
def resolve_ai_skill(
|
||||
message: str,
|
||||
history_messages: Optional[Iterable[dict]] = None,
|
||||
requested_skill_id: Optional[str] = None,
|
||||
has_image: bool = False,
|
||||
) -> AiSkill:
|
||||
explicit_skill = get_ai_skill(requested_skill_id)
|
||||
if explicit_skill:
|
||||
return explicit_skill
|
||||
|
||||
combined_text = f"{_history_text(history_messages or [])}\n{message or ''}".strip()
|
||||
combined_text = combined_text or ""
|
||||
|
||||
best_skill = SKILL_BY_ID["ps-generalist"]
|
||||
best_score = -1
|
||||
|
||||
for skill in SKILLS:
|
||||
score = _score_skill(skill, combined_text, has_image)
|
||||
if score > best_score:
|
||||
best_skill = skill
|
||||
best_score = score
|
||||
|
||||
if has_image and best_score <= 1:
|
||||
return SKILL_BY_ID["visual-analysis-advisor"]
|
||||
|
||||
return best_skill
|
||||
File diff suppressed because it is too large
Load Diff
@@ -5,18 +5,65 @@
|
||||
|
||||
from typing import List, Optional
|
||||
from datetime import datetime, timedelta
|
||||
from urllib.parse import urlparse
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy.orm import Session, joinedload
|
||||
from sqlalchemy import desc
|
||||
|
||||
from app.db import get_db
|
||||
from app.core.config import settings
|
||||
from app.core.security import get_current_user
|
||||
from app.models.logs import PltProcessRecord, UserActionLog, PltPiece
|
||||
from app.models.user import User
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
CURRENT_QINIU_DOMAIN = settings.QINIU_DOMAIN.rstrip("/")
|
||||
CURRENT_QINIU_HOST = urlparse(CURRENT_QINIU_DOMAIN).netloc.lower() if CURRENT_QINIU_DOMAIN else ""
|
||||
KNOWN_QINIU_HOST_MARKERS = ("qiniuio.com", "clouddn.com", "qnssl.com")
|
||||
LEGACY_QINIU_HOSTS = {
|
||||
"iovip-z2.qiniuio.com",
|
||||
"t9zfnhhrn.hn-bkt.clouddn.com",
|
||||
}
|
||||
|
||||
|
||||
def normalize_qiniu_url(url: Optional[str]) -> Optional[str]:
|
||||
"""将历史记录中的旧七牛域名切换到当前绑定域名。"""
|
||||
if not url or not CURRENT_QINIU_DOMAIN:
|
||||
return url
|
||||
|
||||
parsed = urlparse(url)
|
||||
host = parsed.netloc.lower()
|
||||
|
||||
if not parsed.scheme or not host or not parsed.path:
|
||||
return url
|
||||
|
||||
if host == CURRENT_QINIU_HOST:
|
||||
return url
|
||||
|
||||
is_qiniu_host = host in LEGACY_QINIU_HOSTS or any(
|
||||
host.endswith(marker) for marker in KNOWN_QINIU_HOST_MARKERS
|
||||
)
|
||||
if not is_qiniu_host:
|
||||
return url
|
||||
|
||||
suffix = parsed.path
|
||||
if parsed.query:
|
||||
suffix = f"{suffix}?{parsed.query}"
|
||||
return f"{CURRENT_QINIU_DOMAIN}{suffix}"
|
||||
|
||||
|
||||
def normalize_record_piece_urls(record: Optional[PltProcessRecord]) -> Optional[PltProcessRecord]:
|
||||
"""在返回详情前修正裁片图片地址，兼容旧域名历史记录。"""
|
||||
if not record or not record.pieces:
|
||||
return record
|
||||
|
||||
for piece in record.pieces:
|
||||
piece.image_url = normalize_qiniu_url(piece.image_url)
|
||||
|
||||
return record
|
||||
|
||||
|
||||
# ==================== 数据模型 ====================
|
||||
|
||||
@@ -197,7 +244,7 @@ async def create_plt_record(
|
||||
record_id=record.id,
|
||||
group_id=piece_data.group_id,
|
||||
size=piece_data.size,
|
||||
image_url=piece_data.image_url,
|
||||
image_url=normalize_qiniu_url(piece_data.image_url),
|
||||
width_px=piece_data.width_px,
|
||||
height_px=piece_data.height_px,
|
||||
width_cm=piece_data.width_cm,
|
||||
@@ -235,28 +282,29 @@ async def get_plt_record_detail(
|
||||
if not record:
|
||||
raise HTTPException(status_code=404, detail="记录不存在")
|
||||
|
||||
return record
|
||||
return normalize_record_piece_urls(record)
|
||||
|
||||
|
||||
@router.get("/plt/history", response_model=List[PltRecordResponse])
|
||||
async def get_plt_history(
|
||||
limit: int = Query(20, ge=1, le=100),
|
||||
days: int = Query(7, ge=1, le=30), # 保留天数，默认7天
|
||||
days: int = Query(0, ge=0, le=3650), # 0 表示不限制时间
|
||||
current_username: str = Depends(get_current_user),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
"""获取用户的PLT处理历史（默认保留7天）"""
|
||||
"""获取用户的 PLT 处理历史，days=0 表示全部记录"""
|
||||
user = db.query(User).filter(User.username == current_username).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="用户不存在")
|
||||
|
||||
# 只返回指定天数内的记录
|
||||
since = datetime.now() - timedelta(days=days)
|
||||
|
||||
records = db.query(PltProcessRecord)\
|
||||
.filter(PltProcessRecord.user_id == user.id)\
|
||||
.filter(PltProcessRecord.created_at >= since)\
|
||||
.order_by(desc(PltProcessRecord.created_at))\
|
||||
|
||||
query = db.query(PltProcessRecord)\
|
||||
.filter(PltProcessRecord.user_id == user.id)
|
||||
|
||||
if days > 0:
|
||||
since = datetime.now() - timedelta(days=days)
|
||||
query = query.filter(PltProcessRecord.created_at >= since)
|
||||
|
||||
records = query.order_by(desc(PltProcessRecord.created_at))\
|
||||
.limit(limit)\
|
||||
.all()
|
||||
|
||||
|
||||
@@ -21,6 +21,25 @@ class UserProfileUpdate(BaseModel):
|
||||
nickname: Optional[str] = None
|
||||
avatar: Optional[str] = None
|
||||
email: Optional[str] = None
|
||||
ai_provider: Optional[str] = None
|
||||
ai_base_url: Optional[str] = None
|
||||
ai_chat_base_url: Optional[str] = None
|
||||
ai_vision_base_url: Optional[str] = None
|
||||
ai_image_base_url: Optional[str] = None
|
||||
ai_api_key: Optional[str] = None
|
||||
ai_model: Optional[str] = None
|
||||
ai_vision_model: Optional[str] = None
|
||||
ai_image_model: Optional[str] = None
|
||||
clear_ai_api_key: bool = False
|
||||
|
||||
|
||||
def mask_api_key(value: Optional[str]) -> Optional[str]:
|
||||
if not value:
|
||||
return None
|
||||
raw = value.strip()
|
||||
if len(raw) <= 8:
|
||||
return "*" * len(raw)
|
||||
return f"{raw[:4]}{'*' * (len(raw) - 8)}{raw[-4:]}"
|
||||
|
||||
# ==================== 用户资料管理 ====================
|
||||
|
||||
@@ -43,6 +62,16 @@ async def get_user_profile(db: Session = Depends(get_db), current_username: str
|
||||
"vip_daily_quota": user.vip_daily_quota,
|
||||
"total_check_in_days": user.total_check_in_days,
|
||||
"consecutive_check_in": user.consecutive_check_in,
|
||||
"ai_provider": user.ai_provider or "ark",
|
||||
"ai_base_url": user.ai_base_url or "",
|
||||
"ai_chat_base_url": user.ai_chat_base_url or "",
|
||||
"ai_vision_base_url": user.ai_vision_base_url or "",
|
||||
"ai_image_base_url": user.ai_image_base_url or "",
|
||||
"ai_model": user.ai_model or "",
|
||||
"ai_vision_model": user.ai_vision_model or "",
|
||||
"ai_image_model": user.ai_image_model or "",
|
||||
"has_ai_api_key": bool(user.ai_api_key),
|
||||
"ai_api_key_masked": mask_api_key(user.ai_api_key),
|
||||
"created_at": user.created_at.isoformat() if user.created_at else None
|
||||
}
|
||||
|
||||
@@ -64,6 +93,28 @@ async def update_user_profile(data: UserProfileUpdate, db: Session = Depends(get
|
||||
user.avatar = data.avatar
|
||||
if data.email is not None:
|
||||
user.email = data.email
|
||||
if data.ai_provider is not None:
|
||||
user.ai_provider = data.ai_provider.strip().lower() or None
|
||||
if data.ai_base_url is not None:
|
||||
user.ai_base_url = data.ai_base_url.strip() or None
|
||||
if data.ai_chat_base_url is not None:
|
||||
user.ai_chat_base_url = data.ai_chat_base_url.strip() or None
|
||||
if data.ai_vision_base_url is not None:
|
||||
user.ai_vision_base_url = data.ai_vision_base_url.strip() or None
|
||||
if data.ai_image_base_url is not None:
|
||||
user.ai_image_base_url = data.ai_image_base_url.strip() or None
|
||||
if data.ai_model is not None:
|
||||
user.ai_model = data.ai_model.strip() or None
|
||||
if data.ai_vision_model is not None:
|
||||
user.ai_vision_model = data.ai_vision_model.strip() or None
|
||||
if data.ai_image_model is not None:
|
||||
user.ai_image_model = data.ai_image_model.strip() or None
|
||||
if data.clear_ai_api_key:
|
||||
user.ai_api_key = None
|
||||
elif data.ai_api_key is not None:
|
||||
trimmed_key = data.ai_api_key.strip()
|
||||
if trimmed_key:
|
||||
user.ai_api_key = trimmed_key
|
||||
|
||||
db.commit()
|
||||
|
||||
|
||||
@@ -1,5 +1,30 @@
|
||||
from pathlib import Path
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
BASE_DIR = Path(__file__).resolve().parents[2]
|
||||
ENV_FILE = BASE_DIR / ".env"
|
||||
PLACEHOLDER_SECRET_VALUES = {
|
||||
"",
|
||||
"your-super-secret-key-change-this",
|
||||
"admin-secret-token-change-this",
|
||||
}
|
||||
|
||||
|
||||
def resolve_database_url(raw_url: str) -> str:
|
||||
"""Resolve relative SQLite paths against the Server directory."""
|
||||
if not raw_url.startswith("sqlite:///"):
|
||||
return raw_url
|
||||
|
||||
sqlite_path = raw_url.removeprefix("sqlite:///")
|
||||
if not sqlite_path or sqlite_path == ":memory:":
|
||||
return raw_url
|
||||
|
||||
path_obj = Path(sqlite_path)
|
||||
if path_obj.is_absolute():
|
||||
return raw_url
|
||||
|
||||
return f"sqlite:///{(BASE_DIR / path_obj).resolve().as_posix()}"
|
||||
|
||||
class Settings(BaseSettings):
|
||||
ENV: str = "development"
|
||||
PROJECT_NAME: str = "DesignerCEP Backend"
|
||||
@@ -25,17 +50,41 @@ class Settings(BaseSettings):
|
||||
QINIU_BUCKET: str = ""
|
||||
QINIU_DOMAIN: str = ""
|
||||
|
||||
# AI 配置 — 通义千问 Qwen（DashScope）
|
||||
AI_API_KEY: str = "" # LLM API Key（OpenAI / DeepSeek 等）
|
||||
AI_BASE_URL: str = "" # LLM API 地址（留空则用 OpenAI 默认）
|
||||
AI_MODEL: str = "qwen3-max-2026-01-23" # 默认模型（通义千问3深度思考）
|
||||
AI_VISION_MODEL: str = "qwen-vl-max-latest" # 视觉分析模型（看图分析成衣）
|
||||
AI_IMAGE_EDIT_MODEL: str = "qwen-image-edit-max-2026-01-16" # 图片编辑/生成模型
|
||||
# AI 配置
|
||||
AI_PROVIDER: str = "ark"
|
||||
AI_API_KEY: str = ""
|
||||
AI_BASE_URL: str = "https://ark.cn-beijing.volces.com/api/v3"
|
||||
AI_CHAT_BASE_URL: str = ""
|
||||
AI_VISION_BASE_URL: str = ""
|
||||
AI_IMAGE_BASE_URL: str = ""
|
||||
AI_MODEL: str = "doubao-seed-2-0-mini-260215"
|
||||
AI_VISION_MODEL: str = "doubao-seed-2-0-mini-260215"
|
||||
AI_IMAGE_EDIT_MODEL: str = "doubao-seedream-5-0-260128"
|
||||
AI_CHAT_MODELS: str = ""
|
||||
AI_VISION_MODELS: str = ""
|
||||
AI_IMAGE_EDIT_MODELS: str = ""
|
||||
|
||||
# AI 配置 — Gemini（第三方代理）
|
||||
GEMINI_API_KEY: str = ""
|
||||
GEMINI_BASE_URL: str = "https://api.apiqik.online"
|
||||
# 兼容旧字段
|
||||
ARK_API_KEY: str = ""
|
||||
ARK_BASE_URL: str = "https://ark.cn-beijing.volces.com/api/v3"
|
||||
|
||||
model_config = SettingsConfigDict(env_file=".env", extra="ignore")
|
||||
model_config = SettingsConfigDict(
|
||||
env_file=ENV_FILE,
|
||||
env_file_encoding="utf-8",
|
||||
extra="ignore"
|
||||
)
|
||||
|
||||
def runtime_warnings(self) -> list[str]:
|
||||
warnings: list[str] = []
|
||||
if not ENV_FILE.exists():
|
||||
warnings.append("未找到 Server/.env，当前使用的是代码默认配置。")
|
||||
if self.SECRET_KEY in PLACEHOLDER_SECRET_VALUES:
|
||||
warnings.append("SECRET_KEY 仍是默认值，请在部署前替换。")
|
||||
if self.ADMIN_TOKEN in PLACEHOLDER_SECRET_VALUES:
|
||||
warnings.append("ADMIN_TOKEN 仍是默认值，请在部署前替换。")
|
||||
if not self.AI_API_KEY and not self.ARK_API_KEY:
|
||||
warnings.append("AI_API_KEY / ARK_API_KEY 未配置；如果走用户自带 Key 模式可忽略此项。")
|
||||
return warnings
|
||||
|
||||
settings = Settings()
|
||||
settings.DATABASE_URL = resolve_database_url(settings.DATABASE_URL)
|
||||
|
||||
@@ -2,8 +2,10 @@ from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker, declarative_base
|
||||
from app.core.config import settings
|
||||
|
||||
# 数据库连接字符串，默认使用 SQLite 本地文件
|
||||
SQLALCHEMY_DATABASE_URL = getattr(settings, "DATABASE_URL", "sqlite:///./designercep.db")
|
||||
# 数据库连接字符串，SQLite 相对路径已在配置层解析为 Server 目录下的绝对路径
|
||||
SQLALCHEMY_DATABASE_URL = getattr(
|
||||
settings, "DATABASE_URL", "sqlite:///./designercep.db"
|
||||
)
|
||||
|
||||
# 创建数据库引擎
|
||||
engine = create_engine(
|
||||
@@ -81,7 +83,8 @@ def seed_data():
|
||||
db.close()
|
||||
|
||||
def ensure_migrations():
|
||||
# 轻量级迁移：为 SQLite 动态添加缺失列
|
||||
# 轻量级迁移：仅作为本地 SQLite 的兜底补列逻辑。
|
||||
# 正式环境请优先使用 Alembic（见 Server/migrations/README.md）。
|
||||
if not SQLALCHEMY_DATABASE_URL.startswith("sqlite"):
|
||||
return
|
||||
with engine.connect() as conn:
|
||||
@@ -152,3 +155,27 @@ def ensure_migrations():
|
||||
add_col("users", "consecutive_check_in", "INTEGER DEFAULT 0")
|
||||
if not has_column("users", "last_check_in_date"):
|
||||
add_col("users", "last_check_in_date", "DATE NULL")
|
||||
if not has_column("users", "ai_provider"):
|
||||
add_col("users", "ai_provider", "VARCHAR(32) NULL")
|
||||
if not has_column("users", "ai_api_key"):
|
||||
add_col("users", "ai_api_key", "TEXT NULL")
|
||||
if not has_column("users", "ai_base_url"):
|
||||
add_col("users", "ai_base_url", "VARCHAR(255) NULL")
|
||||
if not has_column("users", "ai_chat_base_url"):
|
||||
add_col("users", "ai_chat_base_url", "VARCHAR(255) NULL")
|
||||
if not has_column("users", "ai_vision_base_url"):
|
||||
add_col("users", "ai_vision_base_url", "VARCHAR(255) NULL")
|
||||
if not has_column("users", "ai_image_base_url"):
|
||||
add_col("users", "ai_image_base_url", "VARCHAR(255) NULL")
|
||||
if not has_column("users", "ai_model"):
|
||||
add_col("users", "ai_model", "VARCHAR(120) NULL")
|
||||
if not has_column("users", "ai_vision_model"):
|
||||
add_col("users", "ai_vision_model", "VARCHAR(120) NULL")
|
||||
if not has_column("users", "ai_image_model"):
|
||||
add_col("users", "ai_image_model", "VARCHAR(120) NULL")
|
||||
|
||||
# chat_sessions 会话工作流字段
|
||||
if not has_column("chat_sessions", "active_skill_id"):
|
||||
add_col("chat_sessions", "active_skill_id", "VARCHAR(80) NULL")
|
||||
if not has_column("chat_sessions", "workflow_state"):
|
||||
add_col("chat_sessions", "workflow_state", "TEXT NULL")
|
||||
|
||||
@@ -2,18 +2,39 @@ from fastapi import FastAPI, Request
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
import os
|
||||
import logging
|
||||
from app.core.config import settings
|
||||
from app.api.v1 import auth, client, admin, analytics, admin_config, feature, checkin, user_profile, stats, algorithm, logs, ai_chat
|
||||
from app.api.v1 import (
|
||||
auth,
|
||||
client,
|
||||
admin,
|
||||
analytics,
|
||||
admin_config,
|
||||
feature,
|
||||
checkin,
|
||||
user_profile,
|
||||
stats,
|
||||
algorithm,
|
||||
logs,
|
||||
ai_chat,
|
||||
ai_pattern,
|
||||
ai_identify,
|
||||
)
|
||||
from app.db import init_db
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
app = FastAPI(title=settings.PROJECT_NAME)
|
||||
log = logging.getLogger("designercep.startup")
|
||||
|
||||
SERVER_DIR = Path(__file__).resolve().parents[2]
|
||||
ARCHIVES_DIR = SERVER_DIR / "archives"
|
||||
|
||||
# Ensure archives directory exists
|
||||
os.makedirs("archives", exist_ok=True)
|
||||
os.makedirs(ARCHIVES_DIR, exist_ok=True)
|
||||
|
||||
# ========== CORS 配置 ==========
|
||||
IS_DEV = os.getenv("ENV", "development") == "development"
|
||||
IS_DEV = settings.ENV == "development"
|
||||
|
||||
if IS_DEV:
|
||||
# 开发环境：保持宽松
|
||||
@@ -27,9 +48,9 @@ if IS_DEV:
|
||||
)
|
||||
else:
|
||||
# 生产环境：严格配置
|
||||
allowed_origins = os.getenv("ALLOWED_ORIGINS", "").split(",")
|
||||
allowed_origins = settings.ALLOWED_ORIGINS.split(",")
|
||||
print(f"🔒 Running in Production Mode: CORS allowed origins: {allowed_origins}")
|
||||
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=allowed_origins,
|
||||
@@ -37,63 +58,93 @@ else:
|
||||
allow_methods=["GET", "POST", "PUT", "DELETE", "OPTIONS"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
|
||||
# ✅ CEP 环境特殊处理 (Origin: null or cep://)
|
||||
@app.middleware("http")
|
||||
async def cep_cors_middleware(request: Request, call_next):
|
||||
origin = request.headers.get("origin")
|
||||
|
||||
|
||||
# CEP 的 Origin 是 null 或 cep://
|
||||
if origin in ["null", None] or (origin and origin.startswith("cep://")):
|
||||
response = await call_next(request)
|
||||
response.headers["Access-Control-Allow-Origin"] = "*"
|
||||
response.headers["Access-Control-Allow-Credentials"] = "true"
|
||||
response.headers["Access-Control-Allow-Methods"] = "GET, POST, PUT, DELETE, OPTIONS"
|
||||
response.headers["Access-Control-Allow-Methods"] = (
|
||||
"GET, POST, PUT, DELETE, OPTIONS"
|
||||
)
|
||||
response.headers["Access-Control-Allow-Headers"] = "*"
|
||||
return response
|
||||
|
||||
|
||||
return await call_next(request)
|
||||
|
||||
app.include_router(auth.router, prefix=f"{settings.API_V1_STR}/auth", tags=["authentication"])
|
||||
app.include_router(client.router, prefix=f"{settings.API_V1_STR}/client", tags=["client"])
|
||||
|
||||
app.include_router(
|
||||
auth.router, prefix=f"{settings.API_V1_STR}/auth", tags=["authentication"]
|
||||
)
|
||||
app.include_router(
|
||||
client.router, prefix=f"{settings.API_V1_STR}/client", tags=["client"]
|
||||
)
|
||||
app.include_router(admin.router, prefix=f"{settings.API_V1_STR}/admin", tags=["admin"])
|
||||
app.include_router(analytics.router, prefix=f"{settings.API_V1_STR}/analytics", tags=["analytics"])
|
||||
app.include_router(
|
||||
analytics.router, prefix=f"{settings.API_V1_STR}/analytics", tags=["analytics"]
|
||||
)
|
||||
|
||||
# 新增路由
|
||||
app.include_router(admin_config.router, prefix=settings.API_V1_STR, tags=["admin-config"])
|
||||
app.include_router(
|
||||
admin_config.router, prefix=settings.API_V1_STR, tags=["admin-config"]
|
||||
)
|
||||
app.include_router(feature.router, prefix=settings.API_V1_STR, tags=["feature"])
|
||||
app.include_router(checkin.router, prefix=settings.API_V1_STR, tags=["checkin"])
|
||||
app.include_router(user_profile.router, prefix=settings.API_V1_STR, tags=["user-profile"])
|
||||
app.include_router(
|
||||
user_profile.router, prefix=settings.API_V1_STR, tags=["user-profile"]
|
||||
)
|
||||
app.include_router(stats.router, prefix=settings.API_V1_STR, tags=["stats"])
|
||||
app.include_router(algorithm.router, prefix=settings.API_V1_STR, tags=["algorithm"])
|
||||
app.include_router(logs.router, prefix=settings.API_V1_STR, tags=["logs"])
|
||||
app.include_router(ai_chat.router, prefix=settings.API_V1_STR, tags=["ai-chat"])
|
||||
app.include_router(ai_pattern.router, prefix=settings.API_V1_STR, tags=["ai-pattern"])
|
||||
app.include_router(ai_identify.router, prefix=settings.API_V1_STR, tags=["ai-identify"])
|
||||
|
||||
|
||||
# Health Check
|
||||
@app.get("/health")
|
||||
def health_check():
|
||||
return {"status": "healthy", "timestamp": datetime.now().isoformat(), "env": "development" if IS_DEV else "production"}
|
||||
return {
|
||||
"status": "healthy",
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
"env": "development" if IS_DEV else "production",
|
||||
}
|
||||
|
||||
|
||||
# ========== Static Files (Development Only) ==========
|
||||
if IS_DEV:
|
||||
# Mount archives directory for download
|
||||
app.mount("/download", StaticFiles(directory="archives"), name="download")
|
||||
app.mount("/download", StaticFiles(directory=ARCHIVES_DIR), name="download")
|
||||
else:
|
||||
print("ℹ️ Production Mode: Static files are NOT mounted by FastAPI (handled by Caddy/Nginx).")
|
||||
print(
|
||||
"ℹ️ Production Mode: Static files are NOT mounted by FastAPI (handled by Caddy/Nginx)."
|
||||
)
|
||||
|
||||
|
||||
@app.get("/")
|
||||
def read_root():
|
||||
from fastapi.responses import RedirectResponse
|
||||
|
||||
if IS_DEV:
|
||||
# 重定向到 Shell 登录页
|
||||
return RedirectResponse(url="/shell/index.html")
|
||||
return {"message": "DesignerCEP API is running"}
|
||||
|
||||
|
||||
@app.on_event("startup")
|
||||
def on_startup():
|
||||
# 应用启动时初始化数据库（创建表）
|
||||
init_db()
|
||||
for warning in settings.runtime_warnings():
|
||||
log.warning("[Config] %s", warning)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
|
||||
uvicorn.run("app.main:app", host="0.0.0.0", port=8000, reload=True)
|
||||
|
||||
@@ -15,6 +15,8 @@ class ChatSession(Base):
|
||||
user_id = Column(Integer, ForeignKey("users.id"), nullable=False, index=True)
|
||||
username = Column(String(64), nullable=False, index=True)
|
||||
title = Column(String(200), default="新对话") # 对话标题（取首条消息摘要）
|
||||
active_skill_id = Column(String(80), nullable=True)
|
||||
workflow_state = Column(Text, nullable=True)
|
||||
created_at = Column(DateTime(timezone=True), server_default=func.now())
|
||||
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
|
||||
|
||||
|
||||
@@ -32,6 +32,17 @@ class User(Base):
|
||||
vip_expire = Column(DateTime(timezone=True), nullable=True)
|
||||
vip_daily_quota = Column(Integer, default=0)
|
||||
vip_quota_reset_date = Column(Date, nullable=True)
|
||||
|
||||
# User-owned AI credentials (backend forwards requests using the user's own key)
|
||||
ai_provider = Column(String(32), nullable=True)
|
||||
ai_api_key = Column(Text, nullable=True)
|
||||
ai_base_url = Column(String(255), nullable=True)
|
||||
ai_chat_base_url = Column(String(255), nullable=True)
|
||||
ai_vision_base_url = Column(String(255), nullable=True)
|
||||
ai_image_base_url = Column(String(255), nullable=True)
|
||||
ai_model = Column(String(120), nullable=True)
|
||||
ai_vision_model = Column(String(120), nullable=True)
|
||||
ai_image_model = Column(String(120), nullable=True)
|
||||
|
||||
total_check_in_days = Column(Integer, default=0)
|
||||
consecutive_check_in = Column(Integer, default=0)
|
||||
|
||||
28
Server/migrations/README.md
Normal file
28
Server/migrations/README.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# 数据库迁移说明
|
||||
|
||||
当前项目保留两套迁移策略：
|
||||
|
||||
1. `Alembic`
|
||||
适用于正式环境和多人协作，是推荐方案。
|
||||
2. `app/db.py -> ensure_migrations()`
|
||||
仅作为本地 SQLite 开发时的兜底补列逻辑。
|
||||
|
||||
## 推荐流程
|
||||
|
||||
```bash
|
||||
cd Server
|
||||
.venv\Scripts\python -m alembic upgrade head
|
||||
```
|
||||
|
||||
新增模型字段后生成迁移：
|
||||
|
||||
```bash
|
||||
cd Server
|
||||
.venv\Scripts\python -m alembic revision --autogenerate -m "describe_change"
|
||||
```
|
||||
|
||||
注意事项：
|
||||
|
||||
- `Server/alembic/env.py` 已按 `Server` 目录固定加载配置，不依赖当前启动目录。
|
||||
- 自动生成迁移前，先确认 `app/models/` 中新增模型已被 Alembic 导入。
|
||||
- 如果只是本机 SQLite 调试，项目仍会在启动时走一次轻量补列；但不要把它当作正式迁移方案。
|
||||
@@ -20,3 +20,4 @@ Pillow
|
||||
qiniu
|
||||
openai
|
||||
httpx
|
||||
volcengine-python-sdk[ark]
|
||||
|
||||
329
Server/test_identify_prompt.py
Normal file
329
Server/test_identify_prompt.py
Normal file
@@ -0,0 +1,329 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
提示词迭代测试脚本
|
||||
直接调用视觉模型测试 identify_pieces 的提示词效果
|
||||
"""
|
||||
import base64, json, os, sys
|
||||
|
||||
# 加载环境
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from dotenv import load_dotenv
|
||||
load_dotenv()
|
||||
|
||||
from openai import OpenAI
|
||||
from app.core.config import settings
|
||||
|
||||
# ==================== 配置 ====================
|
||||
|
||||
GARMENT_IMG = "debug_images/1770525898_input_0.jpg" # 成衣照片
|
||||
CANVAS_IMG = "debug_images/1770525898_input_1.jpg" # 裁片轮廓
|
||||
|
||||
LAYERS_INFO = """组 1 (group, 4120x5280px, 位于路径: /组 1)
|
||||
图层 1 (LayerKind.NORMAL, 4120x5280px, 位于路径: /组 1/图层 1)
|
||||
组 2 (group, 3999x4561px, 位于路径: /组 2)
|
||||
图层 2 (LayerKind.NORMAL, 3999x4561px, 位于路径: /组 2/图层 2)
|
||||
组 3 (group, 3529x333px, 位于路径: /组 3)
|
||||
图层 3 (LayerKind.NORMAL, 3529x333px, 位于路径: /组 3/图层 3)
|
||||
组 4 (group, 2423x2004px, 位于路径: /组 4)
|
||||
图层 4 (LayerKind.NORMAL, 2423x2004px, 位于路径: /组 4/图层 4)
|
||||
组 5 (group, 2422x2003px, 位于路径: /组 5)
|
||||
图层 1 拷贝 (LayerKind.NORMAL, 2422x2003px, 位于路径: /组 5/图层 1 拷贝)"""
|
||||
|
||||
# ==================== 提示词版本 ====================
|
||||
|
||||
def make_prompt_v2(layers_info: str) -> str:
|
||||
"""v2 提示词：新增 pattern_mode + 不可见部位推理"""
|
||||
# 动态提取图层名
|
||||
layer_names = []
|
||||
for line in layers_info.strip().split('\n'):
|
||||
name = line.split('(')[0].strip()
|
||||
if name:
|
||||
layer_names.append(name)
|
||||
ex1 = layer_names[0] if len(layer_names) > 0 else "图层名1"
|
||||
ex2 = layer_names[1] if len(layer_names) > 1 else "图层名2"
|
||||
ex3 = layer_names[2] if len(layer_names) > 2 else "图层名3"
|
||||
|
||||
return f"""你需要完成三个任务：
|
||||
|
||||
**任务1：识别裁片部位**
|
||||
Image 2 是 PS 文档中的裁片轮廓图。以下是图层信息：
|
||||
{layers_info}
|
||||
|
||||
请根据形状、大小识别每个**图层组**是什么服装部位（前片、后片、袖子、领口等）。
|
||||
⚠️ mapping 的 key 必须是上面图层信息中的**实际图层组名称**（如 "{ex1}"、"{ex2}" 等），不要自己编名字！
|
||||
|
||||
**任务2：判断花型覆盖模式 pattern_mode**
|
||||
Image 1 是成衣照片。请判断这件衣服的花型属于哪种模式：
|
||||
|
||||
- "all_over"：所有裁片来自同一块面料（碎花/格子/渐变等整体花型覆盖全身）
|
||||
- "placement"：各裁片独立处理（有的印花有的纯色，如卡通卫衣）
|
||||
- "color_block"：全部纯色拼接
|
||||
|
||||
如果是 all_over，还需要提供 fabric_info：
|
||||
- fabric_type: 面料印花类型（数码印花/提花/丝网印/...）
|
||||
- pattern_direction: 花型方向（均匀铺满/上下渐变/左右渐变/斜向）
|
||||
- pattern_elements: 花型元素描述
|
||||
- color_transition: 色彩过渡描述
|
||||
- density: 花型密度描述
|
||||
|
||||
**任务3：分析每个裁片的图案类型**
|
||||
对每个裁片分析：
|
||||
- type: solid / fill_pattern / theme_pattern / mixed_pattern / all_over
|
||||
- solid 和 theme_pattern 必须给 color（hex值）
|
||||
- 如果某个裁片在照片中**看不到**（如后片），设置 visible: false，并根据前片推理：
|
||||
- 全身花型 → 后片与前片相同
|
||||
- 局部印花 → 后片通常为底色纯色
|
||||
- 给出推理理由 inference_reason 和置信度 confidence（high/medium/low）
|
||||
- 如果左右对称（如左右袖），标记 mirror_of 字段
|
||||
|
||||
用 JSON 回答（注意：key 用实际图层名）：
|
||||
```json
|
||||
{{
|
||||
"pattern_mode": "all_over",
|
||||
"fabric_info": {{
|
||||
"fabric_type": "数码印花",
|
||||
"pattern_direction": "上下渐变",
|
||||
"pattern_elements": "白色百合花朵 + 灰白格纹底",
|
||||
"color_transition": "从黑色渐变到灰白色",
|
||||
"density": "上部花朵密集大朵，下部稀疏小朵"
|
||||
}},
|
||||
"mapping": {{
|
||||
"{ex1}": "左袖",
|
||||
"{ex2}": "前片"
|
||||
}},
|
||||
"analysis": [
|
||||
{{"layer": "{ex1}", "piece": "左袖", "type": "all_over", "visible": true, "description": "黑底白花渐变"}},
|
||||
{{"layer": "{ex2}", "piece": "前片", "type": "all_over", "visible": true, "description": "上半黑底大花+下半灰白格纹"}},
|
||||
{{"layer": "{ex3}", "piece": "后片", "type": "all_over", "visible": false, "inferred_from": "{ex2}", "inference_reason": "全身花型面料，后片花型与前片相同", "confidence": "high"}}
|
||||
],
|
||||
"symmetric_pairs": [["{ex1}", "其他对称片"]],
|
||||
"base_color": "#1A1A1A"
|
||||
}}
|
||||
```
|
||||
|
||||
⚠️ 重要：
|
||||
- mapping 和 analysis 中的 layer 必须用**实际图层组名称**
|
||||
- 看不到的部位设 visible: false 并说明推理依据
|
||||
- 左右对称的片标记 symmetric_pairs
|
||||
只返回 JSON，不要其他文字。"""
|
||||
|
||||
|
||||
# ==================== 调用模型 ====================
|
||||
|
||||
def call_vision(prompt: str, garment_b64: str, canvas_b64: str, model: str = None):
|
||||
"""调用视觉模型"""
|
||||
use_model = model or settings.AI_VISION_MODEL
|
||||
client = OpenAI(
|
||||
api_key=settings.AI_API_KEY,
|
||||
base_url=settings.AI_BASE_URL or "https://api.openai.com/v1",
|
||||
)
|
||||
|
||||
content = [
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{garment_b64}"}},
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{canvas_b64}"}},
|
||||
{"type": "text", "text": prompt},
|
||||
]
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f"模型: {use_model}")
|
||||
print(f"提示词长度: {len(prompt)} 字符")
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
completion = client.chat.completions.create(
|
||||
model=use_model,
|
||||
messages=[{"role": "user", "content": content}],
|
||||
)
|
||||
|
||||
return completion.choices[0].message.content or ""
|
||||
|
||||
|
||||
def parse_result(raw: str) -> dict:
|
||||
"""解析 JSON 结果"""
|
||||
clean = raw.strip()
|
||||
if "```" in clean:
|
||||
clean = clean.split("```")[1]
|
||||
if clean.startswith("json"):
|
||||
clean = clean[4:]
|
||||
clean = clean.strip()
|
||||
try:
|
||||
return json.loads(clean)
|
||||
except json.JSONDecodeError as e:
|
||||
print(f"⚠️ JSON 解析失败: {e}")
|
||||
print(f"原始回复:\n{raw[:500]}")
|
||||
return {}
|
||||
|
||||
|
||||
def evaluate_result(result: dict) -> list:
|
||||
"""评估结果质量，返回问题列表"""
|
||||
issues = []
|
||||
|
||||
if not result:
|
||||
issues.append("❌ 解析失败，无结果")
|
||||
return issues
|
||||
|
||||
# 检查 pattern_mode
|
||||
mode = result.get("pattern_mode")
|
||||
if not mode:
|
||||
issues.append("❌ 缺少 pattern_mode")
|
||||
elif mode not in ("all_over", "placement", "color_block"):
|
||||
issues.append(f"⚠️ pattern_mode 值异常: {mode}")
|
||||
|
||||
# 检查 mapping 是否用了真实图层名
|
||||
mapping = result.get("mapping", {})
|
||||
for key in mapping:
|
||||
if key.startswith("M-") or key.startswith("Layer"):
|
||||
issues.append(f"❌ mapping key '{key}' 不是真实图层名（应为 '组 X' 格式）")
|
||||
|
||||
# 检查 analysis
|
||||
analysis = result.get("analysis", [])
|
||||
if not analysis:
|
||||
issues.append("❌ 缺少 analysis")
|
||||
else:
|
||||
has_invisible = any(a.get("visible") == False for a in analysis)
|
||||
if not has_invisible:
|
||||
issues.append("⚠️ 没有标记不可见部位（后片通常看不到）")
|
||||
|
||||
has_mirror = any("mirror_of" in a for a in analysis)
|
||||
symmetric = result.get("symmetric_pairs", [])
|
||||
if not has_mirror and not symmetric:
|
||||
issues.append("⚠️ 没有标记对称片（左右袖应该对称）")
|
||||
|
||||
for a in analysis:
|
||||
if a.get("type") in ("solid", "theme_pattern") and not a.get("color"):
|
||||
issues.append(f"⚠️ {a.get('layer')} 类型为 {a['type']} 但缺少 color")
|
||||
|
||||
# 检查 fabric_info（仅 all_over）
|
||||
if mode == "all_over":
|
||||
fi = result.get("fabric_info")
|
||||
if not fi:
|
||||
issues.append("❌ all_over 模式缺少 fabric_info")
|
||||
else:
|
||||
for key in ("fabric_type", "pattern_direction", "pattern_elements", "color_transition", "density"):
|
||||
if not fi.get(key):
|
||||
issues.append(f"⚠️ fabric_info 缺少 {key}")
|
||||
|
||||
if not issues:
|
||||
issues.append("✅ 完美！所有字段齐全")
|
||||
|
||||
return issues
|
||||
|
||||
|
||||
# ==================== 主流程 ====================
|
||||
|
||||
def main():
|
||||
# 修复 Windows GBK 输出
|
||||
import io
|
||||
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', errors='replace')
|
||||
|
||||
# 加载测试图片
|
||||
print("[1] Loading test images...")
|
||||
with open(GARMENT_IMG, "rb") as f:
|
||||
garment_b64 = base64.b64encode(f.read()).decode()
|
||||
with open(CANVAS_IMG, "rb") as f:
|
||||
canvas_b64 = base64.b64encode(f.read()).decode()
|
||||
print(f" garment: {len(garment_b64)//1024}KB, canvas: {len(canvas_b64)//1024}KB")
|
||||
|
||||
# 生成提示词
|
||||
prompt = make_prompt_v2(LAYERS_INFO)
|
||||
|
||||
# 调用模型
|
||||
print("\n[2] Calling vision model...")
|
||||
raw = call_vision(prompt, garment_b64, canvas_b64)
|
||||
|
||||
# 解析
|
||||
print("\n[3] Model response:")
|
||||
print(raw[:2000])
|
||||
print("\n" + "="*60)
|
||||
|
||||
result = parse_result(raw)
|
||||
|
||||
if result:
|
||||
print("\n[4] Parsed result:")
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
|
||||
# 评估
|
||||
print("\n[5] Quality check:")
|
||||
issues = evaluate_result(result)
|
||||
for issue in issues:
|
||||
print(f" {issue}")
|
||||
|
||||
# ==================== 阶段 2: Gemini 生图 ====================
|
||||
if result and result.get("pattern_mode") == "all_over" and result.get("fabric_info"):
|
||||
print("\n" + "="*60)
|
||||
print("[6] Testing Gemini image generation (all_over fabric)...")
|
||||
|
||||
fi = result["fabric_info"]
|
||||
gen_prompt = (
|
||||
f"Please generate a high-resolution flat fabric pattern image based on this garment photo.\n"
|
||||
f"\n"
|
||||
f"Requirements:\n"
|
||||
f"1. Output a RECTANGULAR flat fabric image (not garment shape, just the fabric laid flat)\n"
|
||||
f"2. Fabric type: {fi.get('fabric_type', 'digital print')}\n"
|
||||
f"3. Pattern elements: {fi.get('pattern_elements', 'floral')}\n"
|
||||
f"4. Pattern direction: {fi.get('pattern_direction', 'uniform')}\n"
|
||||
f"5. Color transition: {fi.get('color_transition', 'none')}\n"
|
||||
f"6. Pattern density: {fi.get('density', 'medium')}\n"
|
||||
f"7. Remove all garment wrinkles/shadows - show flat fabric only\n"
|
||||
f"8. The pattern must be clear, colors accurate, suitable for garment piece overlay\n"
|
||||
f"9. Aspect ratio: 3:4\n"
|
||||
f"10. The image should look like a piece of fabric photographed from directly above on a flat surface"
|
||||
)
|
||||
|
||||
print(f" Prompt: {gen_prompt[:200]}...")
|
||||
print(f" Model: gemini-3-pro-image-preview")
|
||||
|
||||
# 用 Gemini OpenAI 兼容方式生图
|
||||
from openai import OpenAI as GeminiClient
|
||||
gemini = GeminiClient(
|
||||
api_key=settings.GEMINI_API_KEY,
|
||||
base_url=f"{settings.GEMINI_BASE_URL}/v1",
|
||||
)
|
||||
|
||||
content_parts = [
|
||||
{"type": "text", "text": gen_prompt},
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{garment_b64}"}},
|
||||
]
|
||||
|
||||
print(" Calling Gemini 3 Pro Image...")
|
||||
import time
|
||||
t0 = time.time()
|
||||
|
||||
completion = gemini.chat.completions.create(
|
||||
model="gemini-3-pro-image-preview",
|
||||
messages=[{"role": "user", "content": content_parts}],
|
||||
)
|
||||
|
||||
elapsed = time.time() - t0
|
||||
resp_content = completion.choices[0].message.content or ""
|
||||
print(f" Response length: {len(resp_content)} chars, time: {elapsed:.1f}s")
|
||||
|
||||
# 提取图片
|
||||
import re
|
||||
match = re.search(r'!\[.*?\]\((data:image/(\w+);base64,([^)]+))\)', resp_content)
|
||||
if not match:
|
||||
match = re.search(r'(data:image/(\w+);base64,([A-Za-z0-9+/=]+))', resp_content)
|
||||
|
||||
if match:
|
||||
img_format = match.group(2)
|
||||
img_b64 = match.group(3)
|
||||
# 修复 padding
|
||||
padding = 4 - len(img_b64) % 4
|
||||
if padding != 4:
|
||||
img_b64 += '=' * padding
|
||||
|
||||
# 保存
|
||||
out_path = f"debug_images/test_fabric_allover.{img_format}"
|
||||
with open(out_path, "wb") as f:
|
||||
f.write(base64.b64decode(img_b64))
|
||||
print(f" [OK] Fabric image saved: {out_path} ({len(img_b64)//1024}KB)")
|
||||
else:
|
||||
print(f" [FAIL] No image in response")
|
||||
if resp_content:
|
||||
print(f" Response preview: {resp_content[:300]}")
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print("Done.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user