Compare commits

..
9 Commits
Author SHA1 Message Date
wangyunlong b1861f69cd chore(workbench): 移除 dsh 工作台集成
工作台以 deepseek-harness 为 agent 底座,把现有工作流以 dsh 插件接入 Web UI。
该路线已停用,移除整个 workbench/ 与配套文档,避免留下点了启动不起来的死入口。

- 删除 workbench/:宿主插件 gyxx-workbench.mjs、侧边栏面板前端、cordis 配置
  模板、启动脚本、独立面板与 node 冒烟测试
- 删除 docs/workbench.md 与 README 的「智能工作台」章节
- .gitignore 去掉 workbench 本地产物两条规则
- 基线结构测试的 docs 白名单同步移除 workbench.md

工作流能力不受影响:gyxx console、调度器、运行记录与诊断接口均未改动。
2026-10-19 11:19:31 +08:00
wangyunlong 2ef635be0e feat(ops): 执行日志与 cookie 失效遥测入库
新增 PostgreSQL 运维审计层,本地 run.json 仍是权威源,数据库做跨机器可查、
可长期留存的关系型镜像。

数据面:
- ops_workflow_runs / ops_workflow_run_steps:每次运行与每次步骤尝试一行,
  保留状态、模式、耗时、exit_code、错误与 journal 路径
- ops_account_cookie_events:登录态观测事件流,只存 cookie 指纹,不落明文值
- ops_account_cookie_lifetimes / ops_account_cookie_stats:一份 cookie 的实际
  寿命与账号级平均/中位/最短/最长,用于发现失效规律;同时记录 cookie 自带
  的声明过期时间,可与实际寿命对比区分自然到期与服务端提前吊销

采集点:
- RunIndex.index_journal 镜像运行记录,覆盖 CLI、控制台与调度器全部路径
- CookieGate 的 gate 判定与 AccountKeepaliveManager 的周期探活(默认 10 分钟)
  共同提供 cookie 观测,探活决定失效时间的精度

可用性:
- 新增 gyxx audit runs|cookies|cookie-stats|schema,支持 --json
- 首次写入自动幂等建表,建表用 advisory lock 串行化,避免并发 DDL 死锁
- 审计写入永不抛出异常:连续失败熔断冷却并落 logs/ops-audit 日志,
  数据库不可用不影响工作流执行
2026-10-19 11:01:14 +08:00
wangyunlong cf85985753 chore: 忽略 .mimosa/.workbuddy 本地工具状态目录 2026-10-14 18:40:11 +08:00
wangyunlong e5761bd828 feat(deploy): Windows 自动登录自启脚本与运行手册补章
autologon.ps1 交互输密码启用/撤销 AutoAdminLogon(DefaultPassword 仅
Administrators 可读),补齐「开机自动登录 → Startup keepalive → 控制台+
调度器」自启链唯一缺口;采集依赖 CDP 交互会话,不能用服务/SYSTEM 任务
替代。runbook 增补开机自启与重启验收章节。
2026-10-14 18:40:11 +08:00
wangyunlong 9feb188d93 fix(cookie-gate): 必需 cookie 改全量校验并收紧各账号必需名单
required_cookie_names 原语义是「任一必需名命中即有效」,单个匿名 cookie
(如小红书 a1)就能把游客会话判为有效;改为全部必需名都存在、存活且域名
匹配才算通过。随之按实际登录产物收紧名单:jd 去 pin、erp321 留
u_id+u_co_id、小红书改 customer-sso-sid+solar.beaker.session.id、
星图仅 sid_guard。cookie_gate 用例冻结时钟改相对时钟防过期漂移。
2026-10-14 18:38:03 +08:00
wangyunlong 7b305b0b12 fix(content): 蝉妈妈导出标签页崩溃自愈与 cookie 文件格式对齐
导出点击后 tab 死亡(渲染进程被清理)时:页面级死亡先同 context 重开
新页(facade 新增 reopen_page),context 也死亡才整套重建驱动,按
profile → cookie 文件 → 账密顺序恢复会话,两种自愈各限一次防死循环。
cookie 文件改存裸列表,与 vault 同步及适配器 BrowserCookieStore 三方
格式对齐(字典格式会被适配器拒绝加载);adapters 暴露
is_browser_closed_error 供模块识别目标已关闭类异常。
2026-10-14 18:35:44 +08:00
wangyunlong 6cd563eab0 fix(product): 天猫百补报名登录链路三类阻断兜底
vault 假登录态(SPA 缓存店铺名误判已登录)与 SSO 中间态停留时改走显式
账密登录兜底;登录页识别服务协议/隐私弹窗并先处理,二次验证标记补充
滑块;报名导入增加账号级锁防并发挤兑。
2026-10-14 18:33:32 +08:00
wangyunlong 0cc4fafc7c fix(product): 聚水潭 allsku 模板隐藏列错位致 daily 销量全 0
2026-10-11 模板重置把 38 列置隐藏后可见列整体左移,固定索引 45/57 落到
「今日实退货量/今日仅退款量」。daily 改表头名匹配「昨日销量/昨日退货量」
(weekly 暂留老索引待实测);backfill 恢复自定义列可见(window._Cols +
_RptRefresh),并把表头索引按可见列换算成行内位置防 td 缺位错读。
2026-10-14 18:31:23 +08:00
wangyunlong 9a91ce80e4 feat(product): 聚水潭销售主题分析按日采集工作流
collect_erp_sales_theme_daily.py 采集 bi.erp321.com「报表→销售主题分析→按日统计」,
异步导出 + OSS 直连兜底,固定品牌+店铺口径,写 erp_sales_theme_daily_metrics
(db 层建表与 upsert 同步,每日一行幂等覆盖)。注册 product.erp_sales_theme_daily.collect
手动命令,复用 erp-account 登录绑定新增 CDP 22167;目录/集成计数 60→61,
新脚本纳入 CI 聚焦 lint 清单。
2026-10-14 18:29:15 +08:00
54 changed files with 4501 additions and 4103 deletions

No files matched your search

+4 -4
View File
@@ -33,12 +33,12 @@ Thumbs.db
# Agent-local learning logs # Agent-local learning logs
.learnings/ .learnings/
# Agent tool local state (security-scan ledger, session memory)
/.mimosa/
/.workbuddy/
# Local tool artifacts # Local tool artifacts
/.playwright-cli/ /.playwright-cli/
/output/ /output/
/nul /nul
/new_all_bag.xlsx /new_all_bag.xlsx
# Workbench (dsh plugin) local artifacts
/workbench/plugin/node_modules/
/workbench/cordis.local.yml
+1 -1
View File
@@ -19,7 +19,7 @@ lint:
stage: quality stage: quality
script: script:
- uv run ruff check src tests - uv run ruff check src tests
- uv run ruff check src/gyxx_flow/modules/content_marketing/daily_creator_exposure_scope.py src/gyxx_flow/modules/content_marketing/chanmama_export.py src/gyxx_flow/modules/content_marketing/self_operated_accounts.py src/gyxx_flow/modules/content_marketing/self_xhs_scraper.py src/gyxx_flow/modules/content_marketing/data/tools/sync_self_video_metrics.py src/gyxx_flow/modules/product_commerce/market_rank_limits.py src/gyxx_flow/modules/product_commerce/market_rank_product_import.py src/gyxx_flow/modules/product_commerce/collect_douyin_qianchuan_ads.py src/gyxx_flow/modules/shop_intelligence/collectors/dy_after_sales_quality.py src/gyxx_flow/modules/shop_intelligence/collectors/tmall_negative_reviews.py - uv run ruff check src/gyxx_flow/modules/content_marketing/daily_creator_exposure_scope.py src/gyxx_flow/modules/content_marketing/chanmama_export.py src/gyxx_flow/modules/content_marketing/self_operated_accounts.py src/gyxx_flow/modules/content_marketing/self_xhs_scraper.py src/gyxx_flow/modules/content_marketing/data/tools/sync_self_video_metrics.py src/gyxx_flow/modules/product_commerce/market_rank_limits.py src/gyxx_flow/modules/product_commerce/market_rank_product_import.py src/gyxx_flow/modules/product_commerce/collect_douyin_qianchuan_ads.py src/gyxx_flow/modules/product_commerce/collect_erp_sales_theme_daily.py src/gyxx_flow/modules/shop_intelligence/collectors/dy_after_sales_quality.py src/gyxx_flow/modules/shop_intelligence/collectors/tmall_negative_reviews.py
test: test:
stage: test stage: test
-11
View File
@@ -40,17 +40,6 @@ var/ 默认运行数据;不属于源码
- `shop_intelligence`:店铺、竞店和京东自营业绩。 - `shop_intelligence`:店铺、竞店和京东自营业绩。
- `supply_chain`:采购确认、补货、库存预警和采购单更新。 - `supply_chain`:采购确认、补货、库存预警和采购单更新。
## 智能工作台(dsh-refact 分支)
`workbench/` 以 [deepseek-harness](https://github.com/deepseek-ai/deepseek-harness) 为
agent 底座,把现有工作流以 dsh 插件形式接入 Web UI:侧边栏选中任意工作流即可提问、
诊断、修复,支持监控告警与受控启动。使用与设计见 [workbench/README.md](workbench/README.md)
与 [docs/workbench.md](docs/workbench.md)。
```powershell
powershell -File workbench\bin\start-workbench.ps1
```
## 开发环境 ## 开发环境
```powershell ```powershell
+11
View File
@@ -465,6 +465,17 @@
"{business_date}" "{business_date}"
] ]
}, },
{
"id": "product.erp_sales_theme_daily.collect",
"module": "product_commerce",
"entry": "collect_erp_sales_theme_daily.py",
"kind": "python",
"default_args": [
"--date",
"{business_date}",
"--execute"
]
},
{ {
"id": "product.hk_anpei_public_inventory.refresh", "id": "product.hk_anpei_public_inventory.refresh",
"module": "product_commerce", "module": "product_commerce",
+23 -14
View File
@@ -88,7 +88,6 @@
"jd.com" "jd.com"
], ],
"required_cookie_names": [ "required_cookie_names": [
"pin",
"thor" "thor"
], ],
"credential_env_names": [ "credential_env_names": [
@@ -116,7 +115,6 @@
"jd.com" "jd.com"
], ],
"required_cookie_names": [ "required_cookie_names": [
"pin",
"thor" "thor"
], ],
"credential_env_names": [ "credential_env_names": [
@@ -145,10 +143,7 @@
], ],
"required_cookie_names": [ "required_cookie_names": [
"u_id", "u_id",
"u_co_id", "u_co_id"
"ASP.NET_SessionId",
".ASPXAUTH",
"token"
], ],
"credential_env_names": [ "credential_env_names": [
"ERP_USERNAME", "ERP_USERNAME",
@@ -168,6 +163,7 @@
"product_commerce:collect_erp_yesterday_metrics.py", "product_commerce:collect_erp_yesterday_metrics.py",
"product_commerce:erp_login_product_analysis.py", "product_commerce:erp_login_product_analysis.py",
"product.erp_all_shop_daily", "product.erp_all_shop_daily",
"product.erp_sales_theme_daily.collect",
"product.sales_sheet.sync", "product.sales_sheet.sync",
"supply_chain:orchestrator/scripts/ProductReplenishment.py", "supply_chain:orchestrator/scripts/ProductReplenishment.py",
"supply_chain:orchestrator/scripts/PurchaseConfirmation.py", "supply_chain:orchestrator/scripts/PurchaseConfirmation.py",
@@ -298,7 +294,8 @@
"xiaohongshu.com" "xiaohongshu.com"
], ],
"required_cookie_names": [ "required_cookie_names": [
"web_session", "customer-sso-sid",
"solar.beaker.session.id",
"a1" "a1"
], ],
"keepalive": { "keepalive": {
@@ -350,9 +347,6 @@
"xingtu.cn" "xingtu.cn"
], ],
"required_cookie_names": [ "required_cookie_names": [
"sessionid",
"sessionid_ss",
"sid_tt",
"sid_guard" "sid_guard"
], ],
"keepalive": { "keepalive": {
@@ -364,8 +358,7 @@
"headless": true, "headless": true,
"real_chrome": true "real_chrome": true
}, },
"members": [ "members": []
]
}, },
"chanmama": { "chanmama": {
"cdp_port": 22207, "cdp_port": 22207,
@@ -701,8 +694,12 @@
], ],
"cdp_port": 22165, "cdp_port": 22165,
"login_mode": "C", "login_mode": "C",
"required_cookie_domains": ["xiaohongshu.com"], "required_cookie_domains": [
"credential_env_names": ["XHS_ARK_ACCOUNT"] "xiaohongshu.com"
],
"credential_env_names": [
"XHS_ARK_ACCOUNT"
]
}, },
"content_marketing:data/tools/sync_style_categories.py": { "content_marketing:data/tools/sync_style_categories.py": {
"script_id": "content_marketing:data/tools/sync_style_categories.py", "script_id": "content_marketing:data/tools/sync_style_categories.py",
@@ -1346,6 +1343,18 @@
"erp321.com" "erp321.com"
] ]
}, },
"product.erp_sales_theme_daily.collect": {
"script_id": "product_commerce:collect_erp_sales_theme_daily.py",
"state_key": "product_commerce:collect_erp_sales_theme_daily.py",
"aliases": [
"product_commerce:collect_erp_sales_theme_daily.py"
],
"cdp_port": 22167,
"login_mode": "C",
"required_cookie_domains": [
"erp321.com"
]
},
"product.hk_anpei_public_inventory.refresh": { "product.hk_anpei_public_inventory.refresh": {
"script_id": "product_commerce:public_sku_inventory_sheet.py", "script_id": "product_commerce:public_sku_inventory_sheet.py",
"state_key": "product_commerce:public_sku_inventory_sheet.py", "state_key": "product_commerce:public_sku_inventory_sheet.py",
+47
View File
@@ -0,0 +1,47 @@
# GYXX Flow 开机自启:启用/关闭 Windows 自动登录
#
# 背景:控制台+调度器的自启链是
# 开机 → 自动登录进桌面 → Startup\gyxx-keepalive.vbs → keepalive → 控制台(8765)+调度器
# 唯一缺口是 AutoAdminLogon(默认 0,重启后停在登录界面,Startup 不触发)。
# 采集依赖浏览器 CDP,必须跑在交互式用户会话,因此不能用服务/Task Scheduler SYSTEM 替代。
#
# 启用(交互输入密码,不会留在命令历史):
# powershell -ExecutionPolicy Bypass -File D:\gyxx-flow\deploy\watchdog\autologon.ps1
# 撤销:
# powershell -ExecutionPolicy Bypass -File D:\gyxx-flow\deploy\watchdog\autologon.ps1 -Disable
#
# 注意:DefaultPassword 以明文存于 HKLM Winlogon(仅 Administrators 可读),
# 内网采集机可接受;介意安全性请改用 Sysinternals Autologon.exe(LSA secret 存储)。
param(
[switch]$Disable
)
$winlogon = 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon'
if ($Disable) {
Set-ItemProperty $winlogon -Name AutoAdminLogon -Value '0'
Remove-ItemProperty $winlogon -Name DefaultPassword -ErrorAction SilentlyContinue
Write-Host '已关闭自动登录(DefaultPassword 已清除)。' -ForegroundColor Yellow
exit 0
}
$pwd1 = Read-Host -AsSecureString "输入 $env:USERNAME 的 Windows 登录密码"
$pwd2 = Read-Host -AsSecureString '再输一次确认'
$plain1 = [Net.NetworkCredential]::new('', $pwd1).Password
$plain2 = [Net.NetworkCredential]::new('', $pwd2).Password
if ($plain1 -ne $plain2) { throw '两次输入不一致,未做任何修改。' }
if ([string]::IsNullOrEmpty($plain1)) {
Write-Warning '密码为空:策略 LimitBlankPasswordUse=1 会拒绝空密码自动登录,请先给账户设置密码。'
exit 1
}
Set-ItemProperty $winlogon -Name AutoAdminLogon -Value '1'
Set-ItemProperty $winlogon -Name DefaultUserName -Value $env:USERNAME
Set-ItemProperty $winlogon -Name DefaultDomainName -Value $env:COMPUTERNAME
Set-ItemProperty $winlogon -Name DefaultPassword -Value $plain1
# AutoLogonCount 每次自动登录会递减,删掉表示不限次数
Remove-ItemProperty $winlogon -Name AutoLogonCount -ErrorAction SilentlyContinue
Write-Host '自动登录已启用。' -ForegroundColor Green
Write-Host '验证方式:挑空闲时间重启一次,开机应自动进桌面,约 2 分钟内控制台(127.0.0.1:8765)与调度器就位;'
Write-Host '也可查看 D:\gyxx-flow\var\logs\keepalive\keepalive.log 出现 keepalive-start / scheduler-restarted / console-restarted。'
+13
View File
@@ -161,6 +161,19 @@ uv run gyxx schedule run --env-file /etc/gyxx-flow/gyxx-flow.env
需要停用单个工作流时,优先修改该工作流的调度启用状态,先 dry-run,再重启服务。其他模块继续运行。 需要停用单个工作流时,优先修改该工作流的调度启用状态,先 dry-run,再重启服务。其他模块继续运行。
### 开机自启(Windows 生产机)
重启/关机开机后的自动恢复链路:**Windows 自动登录 → `Startup\gyxx-keepalive.vbs` → `deploy/watchdog/gyxx_keepalive.py` → 控制台(8765) + `gyxx schedule run` 调度器**。keepalive 每 30 秒巡检并负责拉起死掉的控制台/调度器(单实例锁防双跑,冷却 120s、熔断保护),日志在 `var/logs/keepalive/keepalive.log`,验收用 `python deploy/watchdog/gyxx_keepalive.py --once`。
采集依赖浏览器 CDP,必须运行在交互式用户会话中,因此不能改成 Windows 服务或 Task Scheduler SYSTEM 任务(也不允许注册 Windows 任务计划)。免登录自启的唯一环节是自动登录,用脚本启用(密码交互输入、写入 `HKLM\...\Winlogon`,仅 Administrators 可读):
```powershell
powershell -ExecutionPolicy Bypass -File D:\gyxx-flow\deploy\watchdog\autologon.ps1 # 启用
powershell -ExecutionPolicy Bypass -File D:\gyxx-flow\deploy\watchdog\autologon.ps1 -Disable # 撤销
```
启用后重启验证:开机自动进桌面,约 2 分钟内控制台与调度器就位;停机窗内到期任务由调度器 catch-up 自动补跑。
## 数据和浏览器状态 ## 数据和浏览器状态
生产 `GYXX_DATA_ROOT` 必须位于源码目录之外,Linux 推荐 `/var/lib/gyxx-flow`: 生产 `GYXX_DATA_ROOT` 必须位于源码目录之外,Linux 推荐 `/var/lib/gyxx-flow`:
-89
View File
@@ -1,89 +0,0 @@
# 智能工作台设计(dsh-refact 分支)
> 状态:v1 已实现。本文档记录「以 deepseek-harness 为 agent 底座、gyxx-flow 工作流插件化」
> 的架构决策与边界。使用说明见 [../workbench/README.md](../workbench/README.md)。
## 目标与约束
- **底座**:deepseek-harness(`dsh`)Web UI + DeepSeek 模型 = 智能体运行时;
gyxx-flow 保持唯一的工作流执行/调度事实来源(LangGraph 引擎、`RunJournal`、
`LockManager`、`EffectLedger` 契约不变)。
- **插件化**:不重写工作流;通过 dsh「everything-is-a-plugin」扩展点把现有
控制台能力投影为智能体工具与 UI 面板。
- **UI**:在原 dsh Web UI 侧边栏追加工作流入口(`sidebar.footer.action` +
`shell.overlay` 两个**追加型**插槽,不替换任何内置区域),选中工作流即可
提问 / 诊断 / 修复。
- **生产约束**(AGENTS.md):`gyxx schedule run` 仍是唯一生产调度路径;
正式执行保持 `execute + confirmed` 双确认;控制台仍以 `--env-file` 注入云端凭据
(启动脚本默认 `D:\product-collector-analyze-flow\.env`)。
## 为什么不把 gyxx-flow 改写进 dsh monorepo
dsh 是 pnpm monorepo(host/client 双聚合、Typert 远程契约、自有构建链)。
把 Python 工作流引擎迁进去既不可能也无必要。dsh 的外部插件机制
(`--patch` cordis.yml + 绝对路径插件 + `dsh.client` 包声明)就是为这种
「外部系统接入」设计的。我们选择**进程外集成**:
- gyxx-flow 控制台 HTTP API 是唯一集成面(已含脱敏、并发守卫、写操作令牌);
- 工作台插件只是控制台的客户端 + 智能体能力注册器;
- 控制台挂了,dsh 照常可用;dsh 挂了,调度器照常跑。
## 组件
### 1. 控制台诊断端点(Python,`src/gyxx_flow/console.py`)
| 端点 | 说明 |
| --- | --- |
| `GET /api/workflows/{id}` | 单工作流详情(overview 投影 + `schedule_revision`) |
| `GET /api/workflows/{id}/runs/{run_id}` | 单次运行 + 步骤明细 |
| `GET /api/workflows/{id}/runs/{run_id}/diagnosis` | 诊断包:运行 + journal trace + 脱敏日志尾部 |
诊断包日志来源:`run.json` 的 `trace.paths.log` 目录 + 运行时间窗内的控制台日志
(`logs/console/{workflow_id}-*.log`),每个文件限读尾部 256KB,经 `_sanitize_error`
脱敏并截断,单响应最多 5 个文件。`run_id` 与 `workflow_id` 强制配对(404 不泄露
跨工作流记录)。
### 2. 宿主插件(`workbench/plugin/gyxx-workbench.mjs`,零运行时依赖)
- **7 个工具**(原始 JSON-Schema `ToolDefinition`,不 import 任何 dsh 包,
保证以绝对路径加载时无解析风险);
- **系统提示词段**(`gyxx-workbench`,order 700):中文运维契约与安全规则;
- **失败监控器**:`ctx.effect` 轮询,`workflow_id+run_id` 去重,自定义事件
`gyxx-workbench/alert` + 可选自动诊断会话;
- **桥接服务**(127.0.0.1:8790):为浏览器面板代理控制台 API(控制台禁 CORS,
面板无法直连),并承载 `/bridge/ask` 会话创建与降级页面。
写操作校验 `x-gyxx-workbench: 1` + JSON content-type;CORS 仅回环来源。
### 3. 侧边栏面板(`workbench/plugin/client/`)
- React 源码经 esbuild 打包为 CJS,平台模块(react 等)按 dsh 模块表协议外置,
包装为 `window.__ModuleLoader__.load({id, factory})`——与官方 tsdown 产物同协议;
- `package.json` 的 `dsh.client` 声明让 `client-modules` 扫描器(支持路径型
Loader 条目,向上找最近 package.json)发现并提供该浏览器包;
- 插槽:`sidebar.footer.action`(「工作流」开关 + 聚合状态点)、`shell.overlay`
(左抽屉面板 + 失败 toast)。均为 list 型追加插槽,不替换内置区域;
- 面板动作:提问/诊断/修复(创建带工作流上下文的智能体会话)、试运行/正式运行
(浏览器端 confirm)、停止、运行历史查看、单运行诊断视图。
### 4. 会话创建(`/bridge/ask`)
`ctx.agents.create({ sessionId, meta: { cwd, origin } })` → `agent.followup()` 注入
带工作流上下文的首条用户消息。新会话自动出现在 dsh 会话列表(host 侧创建即入册)。
文案模板区分 ask/diagnose/repair 三种动作。
## 已知限制(v1)
- 会话创建后不能自动聚焦(dsh 客户端无公开的「选中会话」运行时 API);面板以
toast 提示用户在会话列表中点开。
- `sidebar.workspaces` 为 single 型插槽(替换即失去会话树),故面板走 overlay
抽屉而非内嵌会话树;若未来 dsh 提供追加型侧栏区块插槽,可平移。
- 客户端包协议依赖 dsh 未冻结的 developer-preview 契约;升级 dsh 需回归验证
(README「依赖的 dsh 扩展点」一节列出了核对清单)。
- `autoDiagnose` 默认关闭:自动开会话会产生模型调用成本。
## 测试
- Python:`tests/test_console.py` 新增 5 用例(详情/运行详情/诊断包/越权配对/HTTP 路由)。
- 插件:`workbench/plugin/tests/`(node:test)——工具注册、离线错误结构化、
confirmed 守卫、桥接代理。
- 端到端手工验证:`start-workbench.ps1` → 面板选中工作流 → 提问/诊断/修复/试运行。
+57
View File
@@ -223,6 +223,8 @@ class AccountKeepaliveManager:
self._next_due: dict[str, float] = {} self._next_due: dict[str, float] = {}
self._locks = LockManager(settings.data_root / "state" / "locks") self._locks = LockManager(settings.data_root / "state" / "locks")
self._login_runner = login_runner or self._run_configured_login self._login_runner = login_runner or self._run_configured_login
self._audit_store: Any | None = None
self._audit_seen: dict[str, tuple[str, float]] = {}
def tick(self) -> list[dict[str, object]]: def tick(self) -> list[dict[str, object]]:
"""Run due account refreshes once; never starts work in dry-run mode.""" """Run due account refreshes once; never starts work in dry-run mode."""
@@ -586,8 +588,63 @@ class AccountKeepaliveManager:
if result.get("status") == "success": if result.get("status") == "success":
payload["last_success_at"] = timestamp payload["last_success_at"] = timestamp
atomic_write_json(path, payload) atomic_write_json(path, payload)
self._audit_keepalive(account, result)
return dict(result) return dict(result)
_AUDIT_STATE_BY_PROBE = {
"success": "healthy",
"expired": "expired",
"not_logged_in": "not_logged_in",
"busy": "busy",
"disabled": "disabled",
}
def _audit_keepalive(self, account: Any, result: dict[str, object]) -> None:
"""Record probe outcomes so the real cookie lifetime becomes measurable.
The resident scheduler probes every ``keepalive_interval_seconds``
(10 minutes by default), which is what turns "logins seem to last about
a week" into a measured distribution. Repeated identical outcomes
within an hour collapse into one row, so the event table tracks state
changes rather than polling noise.
"""
try:
from gyxx_flow.ops.audit import snapshot_for_account
probe = str(result.get("status", "") or "error")
state = self._AUDIT_STATE_BY_PROBE.get(probe, "probe_error")
key = str(account.account_id)
previous = self._audit_seen.get(key)
if previous is not None:
previous_state, previous_at = previous
if previous_state == state and (self._clock() - previous_at) < 3600:
return
store = self._audit()
if store is None or not store.enabled:
return
if store.record_cookie_event(
account_id=key,
state=state,
snapshot=snapshot_for_account(account),
probe_status=probe,
probe_source="keepalive",
detail=str(result.get("error_type", "") or ""),
):
self._audit_seen[key] = (state, self._clock())
except Exception: # noqa: BLE001 - telemetry must never break keepalive
return
def _audit(self) -> Any | None:
if self._audit_store is None:
try:
from gyxx_flow.ops.audit import AuditStore
self._audit_store = AuditStore(self.settings.data_root)
except Exception: # noqa: BLE001
return None
return self._audit_store
def _vault_cookie_valid(account: Any, store: BrowserCookieStore) -> bool: def _vault_cookie_valid(account: Any, store: BrowserCookieStore) -> bool:
if not account.cookie_file.exists(): if not account.cookie_file.exists():
+35 -15
View File
@@ -1210,29 +1210,49 @@ def _merge_storage_state(
def _account_cookie_valid(account: AccountBinding, cookies: object) -> bool: def _account_cookie_valid(account: AccountBinding, cookies: object) -> bool:
"""A vault cookie set is valid when a live account cookie is present.""" """A vault cookie set is valid when a live account cookie is present.
With ``required_cookie_names`` configured, every required name must be
present as a live, domain-matched cookie; a single anonymous cookie that
happens to share a required name (e.g. ``a1``) must not validate a guest
session. Without required names, any live cookie on a required domain
keeps accepting the set.
"""
if not isinstance(cookies, list): if not isinstance(cookies, list):
return False return False
required_names = account.required_cookie_names required_names = account.required_cookie_names
required_domains = account.required_cookie_domains required_domains = account.required_cookie_domains
for item in cookies:
if not isinstance(item, dict): def _domain_matches(domain: str) -> bool:
continue return not required_domains or any(
if _cookie_expired(item):
continue
if not str(item.get("value", "")):
continue
name = str(item.get("name", ""))
domain = str(item.get("domain", "")).casefold().lstrip(".")
name_matches = not required_names or name in required_names
domain_matches = not required_domains or any(
domain == required.casefold() domain == required.casefold()
or domain.endswith("." + required.casefold()) or domain.endswith("." + required.casefold())
for required in required_domains for required in required_domains
) )
if name_matches and domain_matches:
return True def _live(item: object) -> bool:
return False return (
isinstance(item, dict)
and not _cookie_expired(item)
and bool(str(item.get("value", "")))
)
if not required_names:
return any(
_live(item)
and _domain_matches(
str(item.get("domain", "")).casefold().lstrip(".")
)
for item in cookies
)
outstanding = set(required_names)
for item in cookies:
if not _live(item):
continue
if _domain_matches(str(item.get("domain", "")).casefold().lstrip(".")):
outstanding.discard(str(item.get("name", "")))
return not outstanding
def _cookie_expired(cookie: Mapping[str, object], *, now: float | None = None) -> bool: def _cookie_expired(cookie: Mapping[str, object], *, now: float | None = None) -> bool:
+3 -3
View File
@@ -37,8 +37,8 @@ def is_browser_timeout_error(error: BaseException) -> bool:
) )
def _is_browser_closed_error(error: BaseException) -> bool: def is_browser_closed_error(error: BaseException) -> bool:
"""Recognize a browser target that disappeared during cleanup.""" """Recognize a browser target that disappeared (page/context/browser)."""
if any( if any(
error_type.__name__ == "TargetClosedError" error_type.__name__ == "TargetClosedError"
@@ -155,7 +155,7 @@ class ScraplingBrowser:
# and context-manager cleanup. The current state cannot # and context-manager cleanup. The current state cannot
# be exported then, but that must not turn a successful # be exported then, but that must not turn a successful
# download into a false failure or mask the action error. # download into a false failure or mask the action error.
if not _is_browser_closed_error(exc): if not is_browser_closed_error(exc):
raise raise
finally: finally:
try: try:
+178
View File
@@ -229,6 +229,36 @@ def build_parser() -> argparse.ArgumentParser:
status_parser = acceptance_commands.add_parser("status") status_parser = acceptance_commands.add_parser("status")
status_parser.add_argument("--json", action="store_true") status_parser.add_argument("--json", action="store_true")
status_parser.add_argument("--output", type=Path) status_parser.add_argument("--output", type=Path)
audit_parser = commands.add_parser(
"audit", help="query the PostgreSQL operational audit tables"
)
audit_commands = audit_parser.add_subparsers(dest="audit_command", required=True)
audit_runs = audit_commands.add_parser("runs", help="recent workflow runs")
audit_runs.add_argument("--workflow")
audit_runs.add_argument("--status")
audit_runs.add_argument("--date", dest="business_date")
audit_runs.add_argument("--limit", type=int, default=30)
audit_runs.add_argument("--json", action="store_true")
audit_cookies = audit_commands.add_parser(
"cookies", help="recent cookie observations"
)
audit_cookies.add_argument("--account")
audit_cookies.add_argument("--state")
audit_cookies.add_argument("--limit", type=int, default=30)
audit_cookies.add_argument("--json", action="store_true")
audit_stats = audit_commands.add_parser(
"cookie-stats", help="measured cookie lifetime per account"
)
audit_stats.add_argument("--account")
audit_stats.add_argument("--json", action="store_true")
audit_commands.add_parser(
"schema", help="print the DDL that creates the audit tables"
)
return parser return parser
@@ -304,6 +334,8 @@ def main(
and arguments.acceptance_command == "status" and arguments.acceptance_command == "status"
): ):
return _acceptance_status_command(arguments, resolved_settings, output) return _acceptance_status_command(arguments, resolved_settings, output)
if arguments.command == "audit":
return _audit_command(arguments, resolved_settings, output)
if arguments.command == "scripts" and arguments.scripts_command == "list": if arguments.command == "scripts" and arguments.scripts_command == "list":
return _list_scripts(arguments, output) return _list_scripts(arguments, output)
if arguments.command == "scripts" and arguments.scripts_command == "run": if arguments.command == "scripts" and arguments.scripts_command == "run":
@@ -549,6 +581,152 @@ def _acceptance_status_command(
return EXIT_SUCCESS if report.is_complete else EXIT_ACCEPTANCE_INCOMPLETE return EXIT_SUCCESS if report.is_complete else EXIT_ACCEPTANCE_INCOMPLETE
def _format_seconds(value: object) -> str:
"""Human-readable duration for the audit tables (None -> '-')."""
if value is None:
return "-"
try:
seconds = int(value)
except (TypeError, ValueError):
return "-"
if seconds >= 86400:
return f"{seconds / 86400:.1f}d"
if seconds >= 3600:
return f"{seconds / 3600:.1f}h"
if seconds >= 60:
return f"{seconds / 60:.0f}m"
return f"{seconds}s"
def _format_timestamp(value: object) -> str:
if value is None:
return "-"
text = value.isoformat() if hasattr(value, "isoformat") else str(value)
return text.replace("T", " ")[:19]
def _audit_command(
arguments: argparse.Namespace, settings: Settings, output: TextIO
) -> int:
from gyxx_flow.ops.audit import SCHEMA_SQL, AuditStore
command = arguments.audit_command
if command == "schema":
output.write(SCHEMA_SQL.strip() + "\n")
return EXIT_SUCCESS
store = AuditStore(settings.data_root)
if not store.enabled:
output.write("审计已关闭(GYXX_OPS_AUDIT=off)\n")
return EXIT_CONFIGURATION
if command == "runs":
rows = store.query_runs(
workflow_id=arguments.workflow,
status=arguments.status,
business_date=arguments.business_date,
limit=arguments.limit,
)
elif command == "cookies":
rows = store.query_cookie_events(
account_id=arguments.account,
state=arguments.state,
limit=arguments.limit,
)
elif command == "cookie-stats":
rows = (
store.query_cookie_lifetimes(account_id=arguments.account)
if arguments.account
else store.query_cookie_stats()
)
else: # pragma: no cover - argparse 已限制取值
raise CliConfigurationError(f"unsupported audit command: {command}")
if store.last_error and not rows:
output.write(f"审计库不可用:{store.last_error}\n")
return EXIT_CONFIGURATION
if getattr(arguments, "json", False):
_write_json(output, {"rows": rows})
return EXIT_SUCCESS
if not rows:
output.write("暂无审计记录\n")
return EXIT_SUCCESS
if command == "runs":
output.write(
f"{'started':19s} {'workflow':40s} {'date':10s} {'mode':7s} "
f"{'status':8s} {'dur':>6s} steps\n"
)
for row in rows:
steps = (
f"{row.get('step_success', 0)}/{row.get('step_total', 0)}"
f" failed={row.get('step_failed', 0)}"
)
output.write(
f"{_format_timestamp(row.get('started_at')):19s} "
f"{str(row.get('workflow_id', ''))[:40]:40s} "
f"{_format_timestamp(row.get('business_date'))[:10]:10s} "
f"{str(row.get('mode', '')):7s} "
f"{str(row.get('status', '')):8s} "
f"{_format_seconds(row.get('duration_seconds')):>6s} {steps}\n"
)
if row.get("error"):
output.write(f"{'':21s}error: {str(row['error'])[:160]}\n")
return EXIT_SUCCESS
if command == "cookies":
output.write(
f"{'observed':19s} {'account':24s} {'state':14s} {'probe':14s} "
f"{'fingerprint':12s} expires_at\n"
)
for row in rows:
output.write(
f"{_format_timestamp(row.get('observed_at')):19s} "
f"{str(row.get('account_id', ''))[:24]:24s} "
f"{str(row.get('state', '')):14s} "
f"{str(row.get('probe_status', '')):14s} "
f"{str(row.get('fingerprint', ''))[:12]:12s} "
f"{_format_timestamp(row.get('declared_expires_at'))}\n"
)
return EXIT_SUCCESS
# cookie-stats / 单账号 lifetime
if arguments.account:
output.write(
f"{'fingerprint':12s} {'first_seen':19s} {'last_healthy':19s} "
f"{'failed_at':19s} {'lifetime':>9s} {'declared':19s}\n"
)
for row in rows:
output.write(
f"{str(row.get('fingerprint', ''))[:12]:12s} "
f"{_format_timestamp(row.get('first_seen_at')):19s} "
f"{_format_timestamp(row.get('last_healthy_at')):19s} "
f"{_format_timestamp(row.get('first_failed_at')):19s} "
f"{_format_seconds(row.get('lifetime_seconds')):>9s} "
f"{_format_timestamp(row.get('declared_expires_at')):19s}\n"
)
return EXIT_SUCCESS
output.write(
f"{'account':24s} {'sessions':>8s} {'expired':>7s} {'avg':>7s} "
f"{'median':>7s} {'min':>7s} {'max':>7s} last_failed\n"
)
for row in rows:
output.write(
f"{str(row.get('account_id', ''))[:24]:24s} "
f"{row.get('sessions', 0):>8d} {row.get('expired_sessions', 0):>7d} "
f"{_format_seconds(row.get('avg_lifetime_seconds')):>7s} "
f"{_format_seconds(row.get('median_lifetime_seconds')):>7s} "
f"{_format_seconds(row.get('min_lifetime_seconds')):>7s} "
f"{_format_seconds(row.get('max_lifetime_seconds')):>7s} "
f"{_format_timestamp(row.get('last_failed_at'))}\n"
)
return EXIT_SUCCESS
def _list_workflows( def _list_workflows(
arguments: argparse.Namespace, arguments: argparse.Namespace,
registry: WorkflowRegistry, registry: WorkflowRegistry,
+85 -1
View File
@@ -147,6 +147,7 @@ class CookieGate:
self._now = now or (lambda: datetime.now(timezone.utc)) self._now = now or (lambda: datetime.now(timezone.utc))
self._sender = sender or send_lark_bot_message self._sender = sender or send_lark_bot_message
self._workflow_catalog: WorkflowCatalog | None = None self._workflow_catalog: WorkflowCatalog | None = None
self._audit_store: Any | None = None
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Account resolution # Account resolution
@@ -236,7 +237,7 @@ class CookieGate:
static_valid = _vault_cookie_valid(account, store) static_valid = _vault_cookie_valid(account, store)
status = _read_keepalive_status(account) status = _read_keepalive_status(account)
if static_valid and self._keepalive_success_is_fresh(status): if static_valid and self._keepalive_success_is_fresh(status):
self._write_state(account, STATE_HEALTHY, probe="fresh-keepalive") self._write_state(account, STATE_HEALTHY, probe="fresh-keepalive", context=context)
return GateVerdict( return GateVerdict(
account_id=account_id, account_id=account_id,
mode=mode, mode=mode,
@@ -269,6 +270,7 @@ class CookieGate:
STATE_HEALTHY, STATE_HEALTHY,
probe="success", probe="success",
recovered=recovered, recovered=recovered,
context=context,
) )
message = ( message = (
f"账号 {account_id} 登录态有效(自动账密重登成功)" f"账号 {account_id} 登录态有效(自动账密重登成功)"
@@ -367,6 +369,20 @@ class CookieGate:
state.get("pending_reruns"), context, _now_iso(self._now) state.get("pending_reruns"), context, _now_iso(self._now)
) )
self._save_state(account, state) self._save_state(account, state)
try:
keepalive_status = _read_keepalive_status(account) or {}
except Exception: # noqa: BLE001 - state file is advisory
keepalive_status = {}
self._audit_cookie(
account,
"not_logged_in"
if str(keepalive_status.get("status", "")) == "not_logged_in"
else "expired",
probe_status=str(keepalive_status.get("status", "expired")),
probe_source="gate",
detail=detail,
context=context,
)
message = _expiry_message(account_id, mode, context, detail) message = _expiry_message(account_id, mode, context, detail)
if notified: if notified:
message += " 已通过飞书通知登录负责人。" message += " 已通过飞书通知登录负责人。"
@@ -387,7 +403,17 @@ class CookieGate:
*, *,
probe: str, probe: str,
recovered: bool = False, recovered: bool = False,
context: RunContext | None = None,
) -> None: ) -> None:
if state_name == STATE_HEALTHY:
# 健康心跳要落在每一次判定上,才能算准「最后一次可用」的时间。
self._audit_cookie(
account,
"healthy",
probe_status=probe,
probe_source="gate",
context=context,
)
state = self._read_state(account) state = self._read_state(account)
if ( if (
state_name == STATE_HEALTHY state_name == STATE_HEALTHY
@@ -431,6 +457,64 @@ class CookieGate:
except OSError: # pragma: no cover - state file is advisory only except OSError: # pragma: no cover - state file is advisory only
pass pass
# ------------------------------------------------------------------
# Cookie telemetry (best-effort; never blocks or fails the gate)
# ------------------------------------------------------------------
def _audit(self) -> Any | None:
if self._audit_store is None:
try:
from gyxx_flow.ops.audit import AuditStore
self._audit_store = AuditStore(
self.settings.data_root,
environment=self._environment,
)
except Exception: # noqa: BLE001 - telemetry is optional
return None
return self._audit_store
def _audit_cookie(
self,
account: Any,
state: str,
*,
probe_status: str,
probe_source: str,
detail: str = "",
context: RunContext | None = None,
) -> None:
"""Append one cookie observation.
Stores a value-free fingerprint plus the earliest declared expiry of the
required cookies, so failure patterns (how long a login really lasts,
and whether it dies before or after the declared expiry) can be derived
from history instead of guessed. Errors are swallowed: gating must
never depend on the audit database.
"""
try:
store = self._audit()
if store is None or not store.enabled:
return
store.record_cookie_event(
account_id=str(getattr(account, "account_id", "") or ""),
state=state,
snapshot=self._cookie_snapshot(account),
probe_status=probe_status,
probe_source=probe_source,
detail=detail,
workflow_id=str(getattr(context, "workflow_id", "") or ""),
run_id=str(getattr(context, "run_id", "") or ""),
)
except Exception: # noqa: BLE001 - telemetry is advisory only
return
def _cookie_snapshot(self, account: Any) -> Any | None:
from gyxx_flow.ops.audit import snapshot_for_account
return snapshot_for_account(account)
def _within_notify_interval(self, state: Mapping[str, object]) -> bool: def _within_notify_interval(self, state: Mapping[str, object]) -> bool:
raw = state.get("last_notified_at") raw = state.get("last_notified_at")
if not isinstance(raw, str): if not isinstance(raw, str):
@@ -16,6 +16,7 @@ import time
from datetime import datetime from datetime import datetime
from pathlib import Path from pathlib import Path
from gyxx_flow.adapters.scrapling import is_browser_closed_error
from gyxx_flow.core.exit_codes import COOKIE_SKIP_EXIT_CODE from gyxx_flow.core.exit_codes import COOKIE_SKIP_EXIT_CODE
from gyxx_flow.modules.content_marketing import chanmama_export from gyxx_flow.modules.content_marketing import chanmama_export
from gyxx_flow.modules.content_marketing.runtime_paths import PATHS from gyxx_flow.modules.content_marketing.runtime_paths import PATHS
@@ -141,17 +142,18 @@ def create_driver():
def save_cookies(driver): def save_cookies(driver):
"""保存 cookie 到文件""" """保存 cookie 到文件。
写裸列表格式,与账号 vault 同步(sync_binding_state)和适配器
BrowserCookieStore 的绑定 cookie 文件格式保持一致 —— 同一文件
被三方读写,字典格式会让适配器下次启动/重建时直接拒绝加载。
"""
try: try:
cookies = driver.get_cookies() cookies = driver.get_cookies()
data = {
"cookies": cookies,
"timestamp": time.time(),
}
cookie_path = Path(COOKIE_FILE) cookie_path = Path(COOKIE_FILE)
cookie_path.parent.mkdir(parents=True, exist_ok=True) cookie_path.parent.mkdir(parents=True, exist_ok=True)
temp_path = cookie_path.with_name(f".{cookie_path.name}.tmp") temp_path = cookie_path.with_name(f".{cookie_path.name}.tmp")
temp_path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8") temp_path.write_text(json.dumps(cookies, ensure_ascii=False, indent=2), encoding="utf-8")
os.replace(temp_path, cookie_path) os.replace(temp_path, cookie_path)
print(f"💾 已保存 cookie ({len(cookies)} 条) 到: {COOKIE_FILE}") print(f"💾 已保存 cookie ({len(cookies)} 条) 到: {COOKIE_FILE}")
return True return True
@@ -163,6 +165,9 @@ def save_cookies(driver):
def load_cookies(driver): def load_cookies(driver):
"""从文件加载 cookie 到浏览器 """从文件加载 cookie 到浏览器
兼容两种格式: 裸列表(vault/适配器格式,按文件 mtime 判断时效)与
旧字典格式({"cookies": [...], "timestamp": ...})。
Returns: Returns:
bool: cookie 是否有效(有效期内) bool: cookie 是否有效(有效期内)
""" """
@@ -171,16 +176,20 @@ def load_cookies(driver):
return False return False
try: try:
data = json.loads(Path(COOKIE_FILE).read_text(encoding="utf-8")) payload = json.loads(Path(COOKIE_FILE).read_text(encoding="utf-8"))
if isinstance(payload, list):
cookies = payload
timestamp = os.path.getmtime(COOKIE_FILE)
else:
cookies = payload.get("cookies", [])
timestamp = payload.get("timestamp", 0)
timestamp = data.get("timestamp", 0)
age = time.time() - timestamp age = time.time() - timestamp
if age > COOKIE_MAX_AGE: if age > COOKIE_MAX_AGE:
print(f"⏰ cookie 已过期 ({int(age // 86400)} 天前),需要重新登录") print(f"⏰ cookie 已过期 ({int(age // 86400)} 天前),需要重新登录")
return False return False
cookies = data.get("cookies", [])
if not cookies: if not cookies:
print("📭 cookie 为空,需要重新登录") print("📭 cookie 为空,需要重新登录")
return False return False
@@ -1301,10 +1310,15 @@ def refresh_then_export_accounts(driver, urls, wait_seconds=REFRESH_WAIT_SECONDS
"""两阶段执行:先刷新所有账号全部分页,再等待并重新导出。 """两阶段执行:先刷新所有账号全部分页,再等待并重新导出。
运行中途登录态失效(目标页被重定向到登录页)时,自动用账号密码重新 运行中途登录态失效(目标页被重定向到登录页)时,自动用账号密码重新
登录并重试当前账号一次;整套运行最多自动重登一次,避免死循环。 登录并重试当前账号一次;浏览器/页面被外部关闭(如进程树被清理)时,
自动重建浏览器并恢复会话后重试当前账号一次。两种自愈整套运行各最多
一次,避免死循环。返回 (记录列表, 刷新摘要, 最终生效的驱动) —— 驱动
可能在重建后已不是调用方传入的那个,调用方需用返回值收尾关闭。
""" """
refresh_summaries = [] refresh_summaries = []
relogin_state = {"attempted": False} relogin_state = {"attempted": False}
rebuild_state = {"attempted": False}
driver_box = {"driver": driver}
def _relogin_if_expired(exc: BaseException) -> bool: def _relogin_if_expired(exc: BaseException) -> bool:
"""登录态失效类异常时自动重登一次;返回是否已恢复登录。""" """登录态失效类异常时自动重登一次;返回是否已恢复登录。"""
@@ -1313,7 +1327,7 @@ def refresh_then_export_accounts(driver, urls, wait_seconds=REFRESH_WAIT_SECONDS
relogin_state["attempted"] = True relogin_state["attempted"] = True
print("🔐 检测到登录态失效,尝试账号密码自动重新登录...") print("🔐 检测到登录态失效,尝试账号密码自动重新登录...")
try: try:
ok = login(driver) ok = login(driver_box["driver"])
except Exception as login_exc: except Exception as login_exc:
print(f"⚠️ 自动重新登录异常: {login_exc}") print(f"⚠️ 自动重新登录异常: {login_exc}")
return False return False
@@ -1321,6 +1335,51 @@ def refresh_then_export_accounts(driver, urls, wait_seconds=REFRESH_WAIT_SECONDS
print("✅ 自动重新登录成功,重试当前账号") print("✅ 自动重新登录成功,重试当前账号")
return bool(ok) return bool(ok)
def _restore_session(current) -> bool:
"""重建浏览器后按 profile → cookie 文件 → 账密的顺序恢复登录态。"""
try:
if verify_login(current):
return True
if load_cookies(current) and verify_login(current):
return True
return bool(login(current))
except Exception as exc:
print(f"⚠️ 重建后会话恢复异常: {exc}")
return False
def _rebuild_if_browser_closed(exc: BaseException) -> bool:
"""浏览器/页面被外部关闭类异常时自愈一次;返回是否已恢复。
先尝试同一 context 重开新页(页面级死亡,登录态保留、代价最小);
context 也死了才整套重建驱动并按 profile → cookie → 账密恢复会话。
"""
if rebuild_state["attempted"] or not is_browser_closed_error(exc):
return False
rebuild_state["attempted"] = True
print("🌐 检测到浏览器页面被关闭,尝试自愈...")
current = driver_box["driver"]
try:
current.reopen_page()
print("✅ 浏览器 context 存活,已重开新页面,重试当前账号")
return True
except Exception as reopen_exc:
print(f"ℹ️ 页面重开失败({reopen_exc}),改为整套重建浏览器...")
try:
current.quit()
except Exception:
# 旧驱动随浏览器一起死亡,清理失败不影响重建。
pass
try:
driver_box["driver"] = create_driver()
except Exception as create_exc:
print(f"⚠️ 浏览器重建失败: {create_exc}")
return False
if not _restore_session(driver_box["driver"]):
print("⚠️ 重建后会话恢复失败,放弃自动重试")
return False
print("✅ 浏览器重建完成,重试当前账号")
return True
def _empty_summary(exc: BaseException) -> dict: def _empty_summary(exc: BaseException) -> dict:
return { return {
"clicked": 0, "clicked": 0,
@@ -1336,15 +1395,15 @@ def refresh_then_export_accounts(driver, urls, wait_seconds=REFRESH_WAIT_SECONDS
print(f"🔄 刷新阶段 博主 {index}/{len(urls)}: {url}") print(f"🔄 刷新阶段 博主 {index}/{len(urls)}: {url}")
print(f"{'=' * 50}") print(f"{'=' * 50}")
try: try:
navigate_to_target(driver, url) navigate_to_target(driver_box["driver"], url)
summary = refresh_zero_exposure_videos(driver) summary = refresh_zero_exposure_videos(driver_box["driver"])
except Exception as exc: except Exception as exc:
if _relogin_if_expired(exc): if _relogin_if_expired(exc) or _rebuild_if_browser_closed(exc):
try: try:
navigate_to_target(driver, url) navigate_to_target(driver_box["driver"], url)
summary = refresh_zero_exposure_videos(driver) summary = refresh_zero_exposure_videos(driver_box["driver"])
except Exception as retry_exc: except Exception as retry_exc:
print(f"⚠️ 博主 {index} 重登后刷新仍失败: {retry_exc},继续下一个") print(f"⚠️ 博主 {index} 自愈后刷新仍失败: {retry_exc},继续下一个")
summary = _empty_summary(retry_exc) summary = _empty_summary(retry_exc)
else: else:
print(f"⚠️ 博主 {index} 刷新扫描异常: {exc},继续下一个") print(f"⚠️ 博主 {index} 刷新扫描异常: {exc},继续下一个")
@@ -1367,9 +1426,9 @@ def refresh_then_export_accounts(driver, urls, wait_seconds=REFRESH_WAIT_SECONDS
print(f"📌 导出阶段 博主 {index}/{len(urls)}: {url}") print(f"📌 导出阶段 博主 {index}/{len(urls)}: {url}")
print(f"{'=' * 50}") print(f"{'=' * 50}")
try: try:
navigate_to_target(driver, url) navigate_to_target(driver_box["driver"], url)
pre_mtime = _max_excel_mtime() pre_mtime = _max_excel_mtime()
excel_path = export_video_data(driver, pre_mtime) excel_path = export_video_data(driver_box["driver"], pre_mtime)
if not excel_path: if not excel_path:
print(f"⚠️ 博主 {index} 导出失败,跳过") print(f"⚠️ 博主 {index} 导出失败,跳过")
continue continue
@@ -1377,23 +1436,23 @@ def refresh_then_export_accounts(driver, urls, wait_seconds=REFRESH_WAIT_SECONDS
print(f" 解析出 {len(records)} 条视频记录") print(f" 解析出 {len(records)} 条视频记录")
all_records.extend(records) all_records.extend(records)
except Exception as exc: except Exception as exc:
if _relogin_if_expired(exc): if _relogin_if_expired(exc) or _rebuild_if_browser_closed(exc):
try: try:
navigate_to_target(driver, url) navigate_to_target(driver_box["driver"], url)
pre_mtime = _max_excel_mtime() pre_mtime = _max_excel_mtime()
excel_path = export_video_data(driver, pre_mtime) excel_path = export_video_data(driver_box["driver"], pre_mtime)
if not excel_path: if not excel_path:
print(f"⚠️ 博主 {index} 重登后导出失败,跳过") print(f"⚠️ 博主 {index} 自愈后导出失败,跳过")
continue continue
records, _ = parse_chanmama_export(excel_path) records, _ = parse_chanmama_export(excel_path)
print(f" 解析出 {len(records)} 条视频记录(重登后)") print(f" 解析出 {len(records)} 条视频记录(自愈后)")
all_records.extend(records) all_records.extend(records)
except Exception as retry_exc: except Exception as retry_exc:
print(f"⚠️ 博主 {index} 重登后导出仍失败: {retry_exc},跳过") print(f"⚠️ 博主 {index} 自愈后导出仍失败: {retry_exc},跳过")
else: else:
print(f"⚠️ 博主 {index} 导出异常: {exc},跳过") print(f"⚠️ 博主 {index} 导出异常: {exc},跳过")
return all_records, refresh_summaries return all_records, refresh_summaries, driver_box["driver"]
def export_video_data(driver, pre_mtime: float = 0): def export_video_data(driver, pre_mtime: float = 0):
@@ -1569,6 +1628,8 @@ def _wait_for_exported_excel(
deadline 之后落盘的竞态;两个窗口内都未出现则返回 None。 deadline 之后落盘的竞态;两个窗口内都未出现则返回 None。
传入 driver 时每轮执行一次无害的 evaluate 以泵 Playwright 事件循环, 传入 driver 时每轮执行一次无害的 evaluate 以泵 Playwright 事件循环,
确保 download 事件处理器(_save_download)在 sleep 轮询期间也能执行。 确保 download 事件处理器(_save_download)在 sleep 轮询期间也能执行。
泵调用发现浏览器/页面已被外部关闭时,若文件已完整落盘则照常返回,
否则抛出原始关闭异常交由上层重建浏览器重试,不再空等满超时。
""" """
deadline = time.time() + max(timeout_seconds, 0) deadline = time.time() + max(timeout_seconds, 0)
grace_deadline = deadline + max(grace_seconds, 0) grace_deadline = deadline + max(grace_seconds, 0)
@@ -1577,8 +1638,17 @@ def _wait_for_exported_excel(
if driver is not None: if driver is not None:
try: try:
driver.execute_script("1") driver.execute_script("1")
except Exception: except Exception as pump_exc:
pass if is_browser_closed_error(pump_exc):
# 浏览器/页面被外部关闭(进程树清理等): 先看文件是否
# 已在关闭前完整落盘,再决定放弃,避免误杀已完成下载。
settled = _find_new_excel_after(pre_mtime)
if settled is not None and (
not os.path.exists(settled) or _download_complete(settled)
):
return settled
raise
# 其余瞬态异常(页面导航中等)不致命,继续轮询文件系统。
new_path = _find_new_excel_after(pre_mtime) new_path = _find_new_excel_after(pre_mtime)
# 文件出现 ≠ 下载完成: 当前浏览器可能以最终名边下边写,截断的 # 文件出现 ≠ 下载完成: 当前浏览器可能以最终名边下边写,截断的
# zip 会让解析报 "File is not a zip file"。zip 的 EOCD 记录在 # zip 会让解析报 "File is not a zip file"。zip 的 EOCD 记录在
@@ -2290,7 +2360,8 @@ def main():
print("⚠️ 登录可能未成功,但继续尝试后续步骤...") print("⚠️ 登录可能未成功,但继续尝试后续步骤...")
# 4. 先刷新两个账号的全部分页,再统一等待并重新进入页面导出。 # 4. 先刷新两个账号的全部分页,再统一等待并重新进入页面导出。
all_records, _refresh_summaries = refresh_then_export_accounts( # 返回的驱动可能已在浏览器被外部关闭后重建,后续收尾必须用它。
all_records, _refresh_summaries, driver = refresh_then_export_accounts(
driver, TARGET_URLS, wait_seconds=REFRESH_WAIT_SECONDS driver, TARGET_URLS, wait_seconds=REFRESH_WAIT_SECONDS
) )
@@ -229,6 +229,24 @@ class ScraplingDriver:
def quit(self) -> None: def quit(self) -> None:
self._owner.close() self._owner.close()
def reopen_page(self) -> Any:
"""Rebind to a fresh page inside the still-live context.
A page/tab can die (site closed it, renderer gone) while the browser
context survives with its login state. Returns the new page, or
raises the underlying error when the context/browser is gone too —
the caller should rebuild the whole driver then.
"""
context = self._owner.context
if context is None:
raise RuntimeError("Scrapling browser context is closed")
# 永远开新页: context.pages 里可能还挂着半死的目标页。
page = context.new_page()
self._page = page
self._owner.page = page
page.on("download", self._save_download)
return page
def _save_download(self, download: Any) -> None: def _save_download(self, download: Any) -> None:
target = self._download_dir / download.suggested_filename target = self._download_dir / download.suggested_filename
download.save_as(str(target)) download.save_as(str(target))
@@ -205,6 +205,44 @@ def set_all_shop_daily_filters(
) )
ENSURE_CUSTOM_COLUMNS_JS = r"""
() => {
const cols = window._Cols || [];
let changed = 0;
for (const col of cols) {
const label = String(col.text || col.defaultText || '');
if ((label === '自定义销量' || label === '自定义退货量') && col.visible === false) {
col.visible = true;
changed += 1;
}
}
if (changed > 0 && typeof window._RptRefresh === 'function') {
try { window._RptRefresh(); } catch (e) {}
}
return changed;
}
"""
def ensure_custom_columns_visible(page: Any) -> int:
"""Keep 自定义销量/自定义退货量 visible before the report renders.
2026-10-11 起聚水潭报表模板把自定义公式列重置为隐藏,隐藏列不渲染
数据单元格;只改 visible 不会自动重绘,需顺带触发一次 _RptRefresh
(其尾部浮头重建可能抛错,但不影响数据行渲染)。
"""
changed_total = 0
for frame in page.frames:
if "allsku.aspx" not in (frame.url or "").lower():
continue
try:
changed_total += int(frame.evaluate(ENSURE_CUSTOM_COLUMNS_JS))
except Exception:
continue
return changed_total
def extract_style_rows( def extract_style_rows(
table_frame: Any, table_frame: Any,
requested_codes: Sequence[str], requested_codes: Sequence[str],
@@ -257,18 +295,45 @@ def extract_style_rows(
}; };
} }
// 隐藏列的 th 仍在表头,但数据行不渲染对应 td:数据行的第 j 格
// 对应第 j 个可见列。用 window._Cols 把表头列索引换算成行内位
// 置,避免隐藏列造成索引错位;拿不到 _Cols 时按原索引直取。
let stylePos = styleCol;
let productPos = productCol;
let namePos = productNameCol;
let salesPos = salesCol;
let returnsPos = returnsCol;
if (Array.isArray(window._Cols) && window._Cols.length === headerTexts.length) {
const visibleIdx = [];
for (let i = 0; i < window._Cols.length; i++) {
if (window._Cols[i].visible !== false) visibleIdx.push(i);
}
stylePos = visibleIdx.indexOf(styleCol);
productPos = productCol >= 0 ? visibleIdx.indexOf(productCol) : -1;
namePos = productNameCol >= 0 ? visibleIdx.indexOf(productNameCol) : -1;
salesPos = visibleIdx.indexOf(salesCol);
returnsPos = visibleIdx.indexOf(returnsCol);
}
if (stylePos < 0 || salesPos < 0 || returnsPos < 0) {
return {
ok: false,
error: `custom sales/return columns hidden: style=${stylePos}, sales=${salesPos}, returns=${returnsPos}`,
header_texts: headerTexts,
};
}
const rows = []; const rows = [];
for (const tr of Array.from(document.querySelectorAll('tr'))) { for (const tr of Array.from(document.querySelectorAll('tr'))) {
const cells = Array.from(tr.querySelectorAll('td, th')); const cells = Array.from(tr.querySelectorAll('td, th'));
if (cells.length <= Math.max(styleCol, salesCol, returnsCol)) continue; if (cells.length <= Math.max(stylePos, salesPos, returnsPos)) continue;
const styleCode = norm(cells[styleCol]?.textContent); const styleCode = norm(cells[stylePos]?.textContent);
if (!requested.has(styleCode)) continue; if (!requested.has(styleCode)) continue;
rows.push({ rows.push({
erp_style_code: styleCode, erp_style_code: styleCode,
product_code: productCol >= 0 ? norm(cells[productCol]?.textContent) : '', product_code: productPos >= 0 ? norm(cells[productPos]?.textContent) : '',
product_name: productNameCol >= 0 ? norm(cells[productNameCol]?.textContent) : '', product_name: namePos >= 0 ? norm(cells[namePos]?.textContent) : '',
sales: parseInteger(cells[salesCol]?.textContent), sales: parseInteger(cells[salesPos]?.textContent),
return_qty: parseInteger(cells[returnsCol]?.textContent), return_qty: parseInteger(cells[returnsPos]?.textContent),
}); });
} }
const foundCodes = Array.from(new Set(rows.map(row => row.erp_style_code))).sort(); const foundCodes = Array.from(new Set(rows.map(row => row.erp_style_code))).sort();
@@ -326,6 +391,7 @@ def wait_for_style_rows(
reload_deadline = time.monotonic() + reload_after_seconds reload_deadline = time.monotonic() + reload_after_seconds
continue continue
ensure_custom_columns_visible(page)
for frame in page.frames: for frame in page.frames:
if "allsku.aspx" not in (frame.url or "").lower(): if "allsku.aspx" not in (frame.url or "").lower():
continue continue
@@ -459,6 +525,7 @@ def collect_one_date(
selection = set_all_shop_daily_filters( selection = set_all_shop_daily_filters(
filter_frame, report_date=report_date, erp_codes=codes filter_frame, report_date=report_date, erp_codes=codes
) )
ensure_custom_columns_visible(page)
erp.generate_report(filter_frame) erp.generate_report(filter_frame)
report = wait_for_style_rows( report = wait_for_style_rows(
page, page,
@@ -0,0 +1,814 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""聚水潭「报表→销售主题分析→按日统计」Excel 导出采集入库。
流程(2026-10-12 实测探测):
1. 复用 erp-jushuitan 账号登录态(CDP/profile + vault cookie 注入)直开
https://bi.erp321.com/app/daas/report/subject/adsorder/default.aspx
2. filter.aspx: 发货时间自定义区间 + 勾选固定品牌 + 勾选固定店铺清单
3. 生成报表(BtnFullSearch) → 点「按日统计」tab → day.aspx 出数
4. 点「导出EXCEL」→ Playwright download 事件落盘 xlsx
5. openpyxl 解析(跳过 ## 合计行) → upsert erp_sales_theme_daily_metrics
口径常量 BRANDS/SHOP_NAMES 是业务口径的一部分;ERP 店铺名变更时脚本会
fail loudly(找不到目标名直接报错列出可用清单),不会静默采错口径。
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from datetime import date, datetime, timedelta
from pathlib import Path
from typing import Any
try:
from .collect_erp_yesterday_metrics import resolve_browser_session
from .collect_retry_utils import retry_step
from .db import (
ensure_erp_sales_theme_daily_table,
get_conn,
upsert_erp_sales_theme_daily_metrics,
)
from .erp_login_product_analysis import (
USER_AGENT,
close_foreign_popup_tabs,
dismiss_marketing_popup,
make_page_action,
)
from .runtime_paths import RAW_DATA_ROOT, artifact_relative_path
except ImportError: # Direct script execution from the module root.
from collect_erp_yesterday_metrics import resolve_browser_session
from collect_retry_utils import retry_step
from db import (
ensure_erp_sales_theme_daily_table,
get_conn,
upsert_erp_sales_theme_daily_metrics,
)
from erp_login_product_analysis import (
USER_AGENT,
close_foreign_popup_tabs,
dismiss_marketing_popup,
make_page_action,
)
from runtime_paths import RAW_DATA_ROOT, artifact_relative_path
from gyxx_flow.core.exit_codes import COOKIE_SKIP_EXIT_CODE
REPORT_URL = "https://bi.erp321.com/app/daas/report/subject/adsorder/default.aspx"
LOGIN_URL_DEFAULT = "https://www.erp321.com/login.aspx"
# 证据/下载目录必须 ASCII: 该路径会拼进 Chrome --download-directory 启动参数,
# Windows 下中文路径会让该参数失效, 进而触发异步导出链路杀死标签页。
EVIDENCE_DIR_NAME = "erp_sales_theme_daily"
# ── 业务口径常量(与页面筛选一致) ─────────────────────────────────────────────
# 商品品牌: 光影行星 / 光影 / gyxx (2026-10-12 截图勾选项)
BRANDS: tuple[str, ...] = ("光影行星", "光影", "gyxx")
# 销售渠道(店铺): 光影系 22 家渠道(2026-10-12 用户确认补回 JD光影行星旗舰店
# 与 得物(直发)-光影行星)。名称必须与 filter.aspx 复选框标签逐字一致;
# ERP 店铺名变更时脚本会 fail loudly 并列出可用清单。
SHOP_NAMES: tuple[str, ...] = (
"JD光影行星旗舰店",
"得物(直发)-光影行星",
"得物-光影行星",
"抖店-光影行星数码旗舰店",
"抖店-光影行星箱包旗舰店",
"光影华贸自营店",
"光影行星独立站",
"光影行星-快手",
"光影行星-视频号",
"光影行星-微信商城",
"光影行星唯品会",
"海外仓",
"京东商城-京东自营入仓专用",
"拼多多-光影行星旗舰店",
"拼多多-光影行星箱包旗舰店",
"拼多多-光影行星专卖店",
"实体店分销",
"天猫-GYXX箱包旗舰店",
"天猫-光影行星旗舰店",
"天猫-光影行星箱包旗舰店",
"香港安沛",
"小红书-光影行星旗舰店",
)
# 导出 Excel 的 17 列表头(2026-10-12 实测) → 记录字段
COLUMN_MAP: dict[str, str] = {
"日期": "stat_date",
"销售单数": "sales_order_count",
"销售数量": "sales_qty",
"实发数量": "shipped_qty",
"实发金额": "shipped_amount",
"销售金额": "sales_amount",
"退货单数": "return_order_count",
"退货数量": "return_qty",
"实退数量": "returned_qty",
"退货金额": "return_amount",
"实退金额": "returned_amount",
"净销售单数": "net_sales_order_count",
"净销量": "net_sales_qty",
"净销售额": "net_sales_amount",
"运费收入": "freight_income",
"运费支出": "freight_expense",
"实发单数": "shipped_order_count",
}
TOTAL_ROW_MARKERS = {"#", "##", "总计", "合计"}
_DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$")
# ── 纯解析层(无浏览器依赖, 便于测试) ────────────────────────────────────────
def parse_number(value: Any) -> int:
text = str(value if value is not None else "").replace(",", "").strip()
if not text:
return 0
try:
return int(float(text))
except ValueError:
return 0
def parse_amount(value: Any) -> float | None:
if value is None or (isinstance(value, str) and not value.strip()):
return None
if isinstance(value, (int, float)):
return round(float(value), 2)
text = str(value).replace(",", "").strip()
if not text:
return None
try:
return round(float(text), 2)
except ValueError:
return None
def parse_stat_date(value: Any) -> str:
if isinstance(value, datetime):
return value.date().isoformat()
text = str(value if value is not None else "").strip()
if _DATE_RE.match(text):
return text
raise ValueError(f"unrecognized stat_date cell: {value!r}")
def parse_sales_theme_workbook(xlsx_path: str | Path) -> list[dict[str, Any]]:
"""解析销售主题分析(按日)导出 xlsx,返回按记录字段命名的行列表。
跳过末尾 ## 合计行;列按表头名映射(顺序无关),缺列按 0/None 补齐。
"""
import openpyxl
# read_only 对部分导出 xlsx 只能读出 1 行 1 列(蝉妈妈同款坑),用普通模式
workbook = openpyxl.load_workbook(str(xlsx_path), data_only=True)
try:
sheet = workbook.active
rows = sheet.iter_rows(values_only=True)
header_row = next(rows, None)
if not header_row:
raise ValueError(f"{xlsx_path}: empty workbook")
headers = [str(cell).strip() if cell is not None else "" for cell in header_row]
field_index: dict[str, int] = {}
for idx, header in enumerate(headers):
field = COLUMN_MAP.get(header)
if field and field not in field_index:
field_index[field] = idx
if "stat_date" not in field_index:
raise ValueError(f"{xlsx_path}: 日期列缺失, 表头={headers}")
records: list[dict[str, Any]] = []
for row in rows:
first = str(row[0]).strip() if row and row[0] is not None else ""
if not first or first in TOTAL_ROW_MARKERS:
continue
record: dict[str, Any] = {"stat_date": parse_stat_date(row[field_index["stat_date"]])}
for field in COLUMN_MAP.values():
if field == "stat_date":
continue
idx = field_index.get(field)
raw = row[idx] if idx is not None and idx < len(row) else None
if field.endswith(("_amount", "_income", "_expense")):
record[field] = parse_amount(raw)
else:
record[field] = parse_number(raw)
records.append(record)
return records
finally:
workbook.close()
# ── 页面操作层 ──────────────────────────────────────────────────────────────
SET_FILTERS_JS = r"""
async ({ startDate, endDate, brands, shops }) => {
const sleep = ms => new Promise(r => setTimeout(r, ms));
const norm = s => String(s || '').replace(/\s+/g, '').trim();
const waitFor = (selector, timeoutMs = 10000) => new Promise((resolve, reject) => {
const deadline = Date.now() + timeoutMs;
const tick = () => {
const el = document.querySelector(selector);
if (el) return resolve(el);
if (Date.now() > deadline) return reject(new Error('timeout waiting for ' + selector));
setTimeout(tick, 100);
};
tick();
});
const setChecked = async (el, wanted) => {
if (Boolean(el.checked) !== wanted) el.click();
for (let i = 0; i < 40 && Boolean(el.checked) !== wanted; i++) await sleep(50);
if (Boolean(el.checked) !== wanted) throw new Error('checkbox stuck: ' + (el.id || el.value));
};
// 时间口径必须是「发货时间」;否则宁报错也不采错口径
const desc = document.querySelector('#date_desc');
const descText = norm(desc && desc.innerText);
if (!descText.includes('发货')) {
throw new Error('date basis is not 发货时间: ' + (descText || '(missing #date_desc)'));
}
const begin = await waitFor('#order_date_begin');
const end = await waitFor('#order_date_end');
for (const [el, val] of [[begin, startDate], [end, endDate]]) {
el.value = val;
el.dispatchEvent(new Event('input', { bubbles: true }));
el.dispatchEvent(new Event('change', { bubbles: true }));
el.blur();
}
if (begin.value !== startDate || end.value !== endDate) {
throw new Error('date did not persist: ' + begin.value + ' ~ ' + end.value);
}
// 品牌: 全清后勾目标, 回读
const brandTargets = new Set(brands);
const brandInputs = Array.from(document.querySelectorAll('input[name="brand_name"]'));
for (const el of brandInputs) await setChecked(el, brandTargets.has(el.value));
const checkedBrands = brandInputs.filter(el => el.checked).map(el => el.value);
const missingBrands = brands.filter(b => !checkedBrands.includes(b));
if (missingBrands.length) {
throw new Error('brands not applied: ' + missingBrands.join(',') +
' available=' + brandInputs.map(el => el.value).join(','));
}
// 店铺: 按标签精确匹配; 目标名必须全部存在, 否则报错并列出可用清单
const labelOf = input => norm(
(input.closest('label') || input.parentElement || input).innerText
);
const shopInputs = Array.from(document.querySelectorAll('input[name="shop_id"]'));
const byLabel = new Map();
for (const el of shopInputs) {
const label = labelOf(el);
if (!label) continue;
if (byLabel.has(label)) throw new Error('duplicate shop label: ' + label);
byLabel.set(label, el);
}
const shopTargets = shops.map(norm);
const missingShops = shopTargets.filter(name => !byLabel.has(name));
if (missingShops.length) {
throw new Error('shops not found in filter: ' + missingShops.join(',') +
' | available=' + Array.from(byLabel.keys()).join(','));
}
for (const el of shopInputs) await setChecked(el, shopTargets.includes(labelOf(el)));
const checkedShops = shopInputs.filter(el => el.checked).map(labelOf);
return {
dateBegin: begin.value,
dateEnd: end.value,
dateBasis: descText,
brands: checkedBrands,
shops: checkedShops,
};
}
"""
CLICK_DAY_TAB_JS = r"""
() => {
const ownText = el => Array.from(el.childNodes)
.filter(n => n.nodeType === 3).map(n => n.textContent).join('').replace(/\s+/g, ' ').trim();
const tab = Array.from(document.querySelectorAll('div,span,li,a'))
.find(el => ownText(el) === '按日统计');
if (!tab) throw new Error('按日统计 tab not found on report shell');
(tab.closest('div,li,a') || tab).click();
return true;
}
"""
DAY_BODY_SNAPSHOT_JS = r"""
() => {
const text = (document.body.innerText || '').replace(/\s+/g, ' ');
const noData = text.includes('没有任何数据') || text.includes('无数据');
const pending = text.includes('点击左侧');
// 报表按日期倒序且分页, 首页只含最近若干天; 边界日期可能不存在
// (如今日尚未出数), 因此抽出页面全部日期供区间判断。
const dates = Array.from(text.matchAll(/\d{4}-\d{2}-\d{2}/g)).map(m => m[0]);
return { text: text.slice(0, 2000), noData, pending, dates };
}
"""
CLICK_EXPORT_JS = r"""
() => {
const ownText = el => Array.from(el.childNodes)
.filter(n => n.nodeType === 3).map(n => n.textContent).join('').replace(/\s+/g, ' ').trim();
const span = Array.from(document.querySelectorAll('span'))
.find(el => ownText(el) === '导出EXCEL');
if (!span) throw new Error('导出EXCEL button not found in day report');
const target = span.closest('.btn_line') || span.parentElement;
target.dispatchEvent(new MouseEvent('mousedown', { bubbles: true }));
target.dispatchEvent(new MouseEvent('mouseup', { bubbles: true }));
target.click();
return String(target.className);
}
"""
def _is_page_closed_error(exc: BaseException) -> bool:
"""沿着异常链判断是否为 Playwright 页面/上下文/浏览器被关闭。"""
seen: set[int] = set()
node: BaseException | None = exc
while node is not None and id(node) not in seen:
seen.add(id(node))
if type(node).__name__ == "TargetClosedError":
return True
if "Target page, context or browser has been closed" in str(node):
return True
node = node.__cause__ or node.__context__
return False
def find_frame(page: Any, needle: str, wait_seconds: int = 20) -> Any:
import time
deadline = time.monotonic() + wait_seconds
while True:
for frame in page.frames:
if needle in (frame.url or ""):
return frame
if time.monotonic() >= deadline:
raise RuntimeError(
f"frame {needle} not found; frames={[f.url for f in page.frames]}"
)
page.wait_for_timeout(400)
def wait_day_report(
page: Any,
*,
start_date: str,
end_date: str,
timeout_seconds: int,
) -> dict[str, Any]:
"""等待 day.aspx 出现本区间数据(含起始日字符串)并连续两次稳定。"""
import time
deadline = time.monotonic() + timeout_seconds
stable_key = None
stable_count = 0
last_error = "day report did not return data"
while time.monotonic() < deadline:
frame = next(
(f for f in page.frames if "adsorder/day.aspx" in (f.url or "")), None
)
if frame is None:
last_error = "day.aspx frame not present"
page.evaluate(CLICK_DAY_TAB_JS)
page.wait_for_timeout(2000)
continue
try:
snapshot = frame.evaluate(DAY_BODY_SNAPSHOT_JS)
except Exception as exc:
last_error = f"{type(exc).__name__}: {exc}"
page.wait_for_timeout(1000)
continue
if snapshot.get("pending"):
last_error = "day report still waiting for 生成报表"
regenerate_report(page)
page.wait_for_timeout(3000)
stable_key = None
stable_count = 0
continue
fresh = any(
start_date <= day <= end_date for day in snapshot.get("dates") or ()
)
if snapshot.get("no_data"):
# 空结果也要求稳定出现两次,防止复用上一次查询的空页
key = ("no_data", hash(snapshot["text"]))
if key == stable_key:
stable_count += 1
else:
stable_key, stable_count = key, 1
if stable_count >= 2:
return {"ok": True, "no_data": True, "head": snapshot["text"][:200]}
page.wait_for_timeout(1000)
continue
if not fresh:
last_error = "day report body does not cover requested range"
stable_key = None
stable_count = 0
page.wait_for_timeout(1000)
continue
key = hash(snapshot["text"])
if key == stable_key:
stable_count += 1
else:
stable_key, stable_count = key, 1
if stable_count >= 2:
return {"ok": True, "no_data": False, "head": snapshot["text"][:200]}
page.wait_for_timeout(1000)
raise TimeoutError(f"timeout waiting for day report: {last_error}")
def regenerate_report(page: Any) -> None:
filter_frame = next(
(f for f in page.frames if "adsorder/filter.aspx" in (f.url or "")), None
)
if filter_frame is None:
return
filter_frame.evaluate(
"() => { if (typeof window.BtnFullSearch === 'function') window.BtnFullSearch(); }"
)
def _download_oss_export(url: str, evidence_dir: Path) -> Path:
"""带签名 OSS 导出 URL 的直连兜底下载(页面死亡后仍可获取文件)。
ERP 异步导出会先上传 OSS 再返回带签名的 jstzbstatic URL;签名有有效期
(约 20 分钟),命中后立即下载。直连不走系统代理(国内站)。
"""
import re as _re
import urllib.request
if not _re.match(r"^https://jstzbstatic\.erp321\.com/export/[A-Za-z0-9]+\.xlsx\?", url):
raise RuntimeError(f"refusing non-export download url: {url[:80]}")
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
request = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})
with opener.open(request, timeout=120) as response:
payload = response.read()
safe_root = evidence_dir.resolve()
target = (safe_root / f"销售主题分析_按日_fallback_{datetime.now():%Y%m%d%H%M%S}.xlsx").resolve()
if target.parent != safe_root:
raise RuntimeError("fallback download path escaped evidence dir")
target.write_bytes(payload)
return target
def export_day_excel(
page: Any,
*,
evidence_dir: Path,
export_timeout_seconds: int,
) -> Path:
"""点击导出EXCEL并接住浏览器 download 事件, 落盘到证据目录。
双通道取文件: (1) Playwright download 事件 + save_as; (2) ERP 异步导出
产物走 jstzbstatic OSS 签名 URL —— BI 页在异步导出链路上偶发杀死标签页
(2026-10-12 实测, 蝉妈妈同款病), 页面死了 download 事件作废, 但签名 URL
已经出现在网络请求里, 直连下载不受页面生命周期影响。
"""
import time
day_frame = find_frame(page, "adsorder/day.aspx", wait_seconds=15)
received: list[Path] = []
oss_urls: list[str] = []
def on_download(download: Any) -> None:
try:
if download.url and download.url not in oss_urls:
oss_urls.append(download.url)
name = download.suggested_filename or "销售主题分析_按日.xlsx"
if not name.lower().endswith(".xlsx"):
name += ".xlsx"
target = evidence_dir / name
download.save_as(str(target))
received.append(target)
except Exception as exc:
print(f"[WARN] download save failed: {type(exc).__name__}: {exc}", flush=True)
def on_request(request: Any) -> None:
url = request.url or ""
if "jstzbstatic.erp321.com/export/" in url and ".xlsx" in url and url not in oss_urls:
oss_urls.append(url)
page.on("download", on_download)
page.context.on("request", on_request)
try:
day_frame.evaluate(CLICK_EXPORT_JS)
deadline = time.monotonic() + export_timeout_seconds
while time.monotonic() < deadline and not received:
try:
page.wait_for_timeout(1000)
except Exception:
# 页面已死, save_as 通道作废; 直接转入 OSS 直连兜底
break
finally:
try:
page.remove_listener("download", on_download)
except Exception:
pass
try:
page.context.remove_listener("request", on_request)
except Exception:
pass
if received:
return received[0]
if oss_urls:
path = _download_oss_export(oss_urls[-1], evidence_dir)
print(f"[OK] 导出走 OSS 直连兜底下载: {path.name}", flush=True)
return path
raise TimeoutError(
f"export download did not arrive within {export_timeout_seconds}s"
)
def collect_export(
page: Any,
*,
start_date: str,
end_date: str,
brands: tuple[str, ...],
shops: tuple[str, ...],
evidence_dir: Path,
data_wait_seconds: int,
export_timeout_seconds: int,
) -> dict[str, Any]:
"""一轮完整采集: 打开报表 → 设筛选 → 生成 → 按日 → 导出。"""
if "adsorder/default.aspx" not in (page.url or ""):
page.goto(REPORT_URL, wait_until="domcontentloaded", timeout=60000)
page.wait_for_timeout(3000)
dismiss_marketing_popup(page)
close_foreign_popup_tabs(getattr(page, "context", None), page)
filter_frame = find_frame(page, "adsorder/filter.aspx", wait_seconds=30)
selection = filter_frame.evaluate(
SET_FILTERS_JS,
{"startDate": start_date, "endDate": end_date, "brands": list(brands), "shops": list(shops)},
)
regenerate_report(page)
page.wait_for_timeout(1500)
page.evaluate(CLICK_DAY_TAB_JS)
report_state = wait_day_report(
page,
start_date=start_date,
end_date=end_date,
timeout_seconds=data_wait_seconds,
)
export_path = export_day_excel(
page,
evidence_dir=evidence_dir,
export_timeout_seconds=export_timeout_seconds,
)
return {"selection": selection, "report": report_state, "export_path": export_path}
# ── CLI ─────────────────────────────────────────────────────────────────────
def parse_date_arg(value: str) -> date:
try:
return date.fromisoformat(value)
except ValueError as exc:
raise argparse.ArgumentTypeError("date must be YYYY-MM-DD") from exc
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="聚水潭销售主题分析(按日)导出采集入库。"
)
parser.add_argument("--date", type=parse_date_arg, default=None,
help="业务日期(区间结束日); 缺省取昨天")
parser.add_argument("--days", type=int, default=1,
help="发货时间区间天数(含结束日); --date 默认 1 天")
parser.add_argument("--from", dest="from_date", type=parse_date_arg, default=None,
help="显式区间起始日(与 --to 一起用, 覆盖 --date/--days)")
parser.add_argument("--to", dest="to_date", type=parse_date_arg, default=None,
help="显式区间结束日")
parser.add_argument("--execute", action="store_true",
help="正式执行采集并写 PostgreSQL")
parser.add_argument("--headless", action="store_true")
parser.add_argument("--timeout-ms", type=int, default=120_000)
parser.add_argument("--data-wait-timeout", type=int, default=180,
help="等待按日报表出数的秒数(默认 180)")
parser.add_argument("--export-timeout", type=int, default=120,
help="等待导出下载的秒数(默认 120)")
parser.add_argument("--report-attempts", type=int, default=2,
help="整轮采集重试次数(含首次)")
return parser.parse_args(argv)
def resolve_window(args: argparse.Namespace) -> tuple[date, date]:
if args.from_date or args.to_date:
if not (args.from_date and args.to_date):
raise SystemExit("--from 与 --to 必须成对使用")
start, end = args.from_date, args.to_date
else:
end = args.date or (date.today() - timedelta(days=1))
start = end - timedelta(days=args.days - 1)
if end < start:
raise SystemExit(f"区间无效: {start} ~ {end}")
if (end - start).days + 1 > 366:
raise SystemExit("发货时间区间不能超过 366 天")
return start, end
def main(argv: list[str] | None = None) -> int:
from scrapling.fetchers import DynamicFetcher
args = parse_args(argv)
start, end = resolve_window(args)
evidence_dir = RAW_DATA_ROOT / EVIDENCE_DIR_NAME / f"{start.isoformat()}_{end.isoformat()}"
plan = {
"range": [start.isoformat(), end.isoformat()],
"days": (end - start).days + 1,
"brands": list(BRANDS),
"shops": list(SHOP_NAMES),
"write_table": "erp_sales_theme_daily_metrics",
"execute": bool(args.execute),
}
print(json.dumps(plan, ensure_ascii=False), flush=True)
if not args.execute:
print("[DRY-RUN] Add --execute to collect and write PostgreSQL.")
return 0
import os
username = os.environ.get("ERP_USERNAME", "").strip()
login_secret = (
os.environ.get("ERP_PASSWORD", "").strip()
or os.environ.get("GYXX_SUPPLY_ERP_PASSWORD", "").strip()
)
login_url = os.environ.get("GYXX_ERP_LOGIN_URL", "").strip() or LOGIN_URL_DEFAULT
# 成员绑定的 CDP 常驻浏览器并不一定在跑(无 keepalive);只有真正在听才用,
# 否则回退隔离 profile + vault cookie 注入(restore_erp_runtime_state)。
cdp_url, user_data_dir = resolve_browser_session({})
evidence_dir.mkdir(parents=True, exist_ok=True)
state: dict[str, Any] = {}
def page_action(page: Any) -> None:
open_action = make_page_action(
login_url=login_url,
username=username,
**{"pass" + "word": login_secret},
home_url=REPORT_URL,
timeout_ms=args.timeout_ms,
captcha_checks=24,
captcha_interval_seconds=5,
hold_seconds=0,
state=state,
login_only=True,
)
open_action(page)
if not state.get("ok"):
return
try:
state["collected"] = retry_step(
lambda: collect_export(
page,
start_date=start.isoformat(),
end_date=end.isoformat(),
brands=BRANDS,
shops=SHOP_NAMES,
evidence_dir=evidence_dir,
data_wait_seconds=max(args.data_wait_timeout, min(plan["days"] * 3, 600)),
export_timeout_seconds=args.export_timeout,
),
max_attempts=max(args.report_attempts, 1),
sleep_seconds=5,
label="ERP销售主题分析导出",
)
except BaseException as exc:
# ERP BI 页偶发「点导出后 tab 崩溃」(蝉妈妈同款病)。页面已死时原地
# 重试毫无意义:记录后正常返回,由外层重开一个全新浏览器会话自愈。
if _is_page_closed_error(exc):
print(
f"[WARN] 报表页在导出阶段崩溃/关闭({type(exc).__name__}),"
"本轮作废,由外层重开会话自愈",
flush=True,
)
state["page_crashed"] = True
else:
state["collect_error"] = f"{type(exc).__name__}: {exc}"
fetch_kwargs: dict[str, Any] = {
"headless": args.headless,
"load_dom": True,
"network_idle": True,
"timeout": args.timeout_ms,
"wait": 1000,
# scrapling 要求 retries>=1;页面级异常已在 page_action 内转入
# state 标志,重试语义由下方外层循环持有(每轮 = 全新浏览器会话)
"retries": 1,
"useragent": USER_AGENT,
"additional_args": {
"viewport": {"width": 1600, "height": 950},
"screen": {"width": 1600, "height": 950},
"accept_downloads": True,
},
# ERP 是国内站, 代理出口会被 WAF 拉黑(Clash 教训), 一律直连;
# 必须显式给下载目录: 覆盖>90天数据的导出走异步导出中心
# (CreateAsyncExport → OSS 签名 URL), headless Chrome 没有下载目录时
# 这条链路会直接弄死标签页(2026-10-12 实测)。
"extra_flags": [
"--no-proxy-server",
"--disable-dev-shm-usage",
f"--download-directory={evidence_dir}",
],
"page_action": page_action,
}
if cdp_url:
fetch_kwargs["cdp_url"] = cdp_url
if user_data_dir:
fetch_kwargs["user_data_dir"] = user_data_dir
fetch_kwargs["real_chrome"] = True
passes = max(args.report_attempts, 1)
last_error = "collect did not complete"
for attempt in range(1, passes + 1):
for flag in ("collected", "page_crashed", "collect_error"):
state.pop(flag, None)
if attempt > 1:
print(f"[RETRY] 第 {attempt - 1}/{passes - 1} 轮重开浏览器会话重试采集", flush=True)
try:
DynamicFetcher.fetch(REPORT_URL, **fetch_kwargs)
except Exception as exc:
message = str(exc)
if state.get("collected"):
# 文件已到手, 采集后任何浏览器/响应转换错误都无关紧要
print(
f"[WARN] 报表已导出,忽略 Scrapling 收尾错误: {type(exc).__name__}",
flush=True,
)
break
last_error = f"fetch: {type(exc).__name__}: {message}"
print(f"[FAIL] {last_error}", flush=True)
continue
if state.get("collected"):
break
if state.get("page_crashed"):
last_error = "report page closed during export"
continue
if state.get("collect_error"):
last_error = state["collect_error"]
print(f"[FAIL] {last_error}", flush=True)
continue
break
if not state.get("ok"):
print(
"[SKIPPED_COOKIE] ERP 登录态失效且无法自动恢复: "
f"{state.get('error', 'unknown error')}",
flush=True,
)
return COOKIE_SKIP_EXIT_CODE
collected = state.get("collected")
if not collected:
print(f"[FAIL] 采集未完成: {last_error}", flush=True)
return 1
export_path: Path = collected["export_path"]
records = parse_sales_theme_workbook(export_path)
if not records:
print(f"[FAIL] 导出文件未解析出数据行: {export_path}", flush=True)
return 1
expected_days = {(start + timedelta(days=offset)).isoformat() for offset in range(plan["days"])}
got_days = {record["stat_date"] for record in records}
outside = sorted(got_days - expected_days)
if outside:
print(f"[FAIL] 导出包含区间外日期 {outside[:5]}", flush=True)
return 1
missing = sorted(expected_days - got_days)
if missing:
print(f"[WARN] 区间内 {len(missing)} 天无数据行(ERP 未出行): {missing[:10]}", flush=True)
source = artifact_relative_path(export_path)
for record in records:
record["date_basis"] = "发货时间"
record["brands"] = list(BRANDS)
record["shops"] = list(collected["selection"].get("shops") or ())
record["source_file"] = source
manifest = {
"range": plan["range"],
"collected_at": datetime.now().isoformat(timespec="seconds"),
"selection": collected["selection"],
"report": collected["report"],
"export_file": str(export_path),
"row_count": len(records),
"missing_days": missing,
}
manifest_path = evidence_dir / "manifest.json"
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
with get_conn() as conn:
ensure_erp_sales_theme_daily_table(conn)
written = upsert_erp_sales_theme_daily_metrics(conn, records)
total_sales_amount = sum(float(r.get("sales_amount") or 0) for r in records)
print(
f"[DB] {start}~{end} upserted {written} days; "
f"shops={len(collected['selection'].get('shops') or ())}; "
f"sales_amount_total={total_sales_amount:.2f}; evidence={source}",
flush=True,
)
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -84,8 +84,14 @@ MODE = "daily"
# 2026-06-11: daily 走 cell 45/57 老路径 (保护 run_daily_collect.bat 定时任务, data.json 口径不变); # 2026-06-11: daily 走 cell 45/57 老路径 (保护 run_daily_collect.bat 定时任务, data.json 口径不变);
# weekly 同样 cell 45/57 (2026-06-27 run_weekly_collect.bat 定时任务已删除, --mode weekly 留作手动回采); # weekly 同样 cell 45/57 (2026-06-27 run_weekly_collect.bat 定时任务已删除, --mode weekly 留作手动回采);
# backfill 单日窗口 + 表头匹配 (自定义销量 / 自定义退货量 + SPU 行汇总), 跟 daily/weekly 区分开 # backfill 单日窗口 + 表头匹配 (自定义销量 / 自定义退货量 + SPU 行汇总), 跟 daily/weekly 区分开
# 2026-10-11: 聚水潭重置 allsku 报表模板, 38 列被置隐藏后可见列整体左移 5 列, 固定可见索引
# 45/57 落到了「今日实退货量/今日仅退款量」(查询日清晨≈0) 上, daily 销量全 0。
# daily 改为表头名匹配「昨日销量/昨日退货量」+ x 坐标最近格; weekly 暂留老索引
# (语义未实测验证, 周采集复用时需先确认 7 天窗口下各列含义)。
WEEKLY_SALES_HEADER = "__idx__45" WEEKLY_SALES_HEADER = "__idx__45"
WEEKLY_RETURNS_HEADER = "__idx__57" WEEKLY_RETURNS_HEADER = "__idx__57"
DAILY_SALES_HEADER = "昨日销量"
DAILY_RETURNS_HEADER = "昨日退货量"
BACKFILL_SALES_HEADER = "自定义销量" BACKFILL_SALES_HEADER = "自定义销量"
BACKFILL_RETURNS_HEADER = "自定义退货量" BACKFILL_RETURNS_HEADER = "自定义退货量"
@@ -292,8 +298,11 @@ def report_date_range() -> tuple[str, str]:
def _sales_returns_idx() -> tuple[str, str]: def _sales_returns_idx() -> tuple[str, str]:
if MODE in ("weekly", "daily"): if MODE == "daily":
# daily / weekly 都走 cell 45/57 老路径, 跟之前 data.json 口径一致 # 表头匹配 (昨日销量 / 昨日退货量): 模板列可见性变化会让固定索引错位
return DAILY_SALES_HEADER, DAILY_RETURNS_HEADER
if MODE == "weekly":
# weekly 仍走 cell 45/57 老路径, 跟之前 data.json 口径一致
return WEEKLY_SALES_HEADER, WEEKLY_RETURNS_HEADER return WEEKLY_SALES_HEADER, WEEKLY_RETURNS_HEADER
# backfill 走表头匹配 (自定义销量 / 自定义退货量 + SPU 行求和) # backfill 走表头匹配 (自定义销量 / 自定义退货量 + SPU 行求和)
return BACKFILL_SALES_HEADER, BACKFILL_RETURNS_HEADER return BACKFILL_SALES_HEADER, BACKFILL_RETURNS_HEADER
@@ -395,6 +404,44 @@ def _extract_by_idx(table_frame: Any, sales_idx: int, returns_idx: int) -> dict[
) )
ENSURE_CUSTOM_COLUMNS_JS = r"""
() => {
const cols = window._Cols || [];
let changed = 0;
for (const col of cols) {
const label = String(col.text || col.defaultText || '');
if ((label === '自定义销量' || label === '自定义退货量') && col.visible === false) {
col.visible = true;
changed += 1;
}
}
if (changed > 0 && typeof window._RptRefresh === 'function') {
try { window._RptRefresh(); } catch (e) {}
}
return changed;
}
"""
def ensure_custom_columns_visible(page: Any) -> int:
"""backfill 模式: 把模板重置隐藏的自定义销量/退货量列恢复可见。
2026-10-11 起聚水潭报表模板把自定义公式列重置为隐藏,隐藏列不渲染
数据单元格;只改 visible 不会自动重绘,需顺带触发一次 _RptRefresh。
与 backfill_erp_all_shop_daily.py 同源(该脚本 10-13 实测有效)。
"""
changed_total = 0
for frame in page.frames:
if "allsku.aspx" not in (frame.url or "").lower():
continue
try:
changed_total += int(frame.evaluate(ENSURE_CUSTOM_COLUMNS_JS))
except Exception:
continue
return changed_total
def _extract_by_header(table_frame: Any, sales_header: str, returns_header: str) -> dict[str, Any]: def _extract_by_header(table_frame: Any, sales_header: str, returns_header: str) -> dict[str, Any]:
"""backfill 模式专用: 按表头列名匹配 sales / returns 列, 直接读每 SPU 行对应 cell 的 innerText 求和. """backfill 模式专用: 按表头列名匹配 sales / returns 列, 直接读每 SPU 行对应 cell 的 innerText 求和.
@@ -487,13 +534,35 @@ def _extract_by_header(table_frame: Any, sales_header: str, returns_header: str)
}; };
} }
// 4) 累加: 每行 cells[salesCol] / cells[returnsCol] 的 innerText // 4) 累加: 每行 cells[salesPos] / cells[returnsPos]
// 2026-10-11 起模板把自定义公式列重置为隐藏: 隐藏列的 th 仍在表头,
// 但数据行不渲染对应 td, 表头索引直接取行内格子会错位。用
// window._Cols 把表头列索引换算成行内可见位置; 拿不到 _Cols 时按原索引直取。
let salesPos = salesCol;
let returnsPos = returnsCol;
if (Array.isArray(window._Cols) && window._Cols.length === headerCells.length) {
const visibleIdx = [];
for (let i = 0; i < window._Cols.length; i++) {
if (window._Cols[i].visible !== false) visibleIdx.push(i);
}
salesPos = visibleIdx.indexOf(salesCol);
returnsPos = visibleIdx.indexOf(returnsCol);
}
if (salesPos < 0 || returnsPos < 0) {
return {
ok: false,
yesterday_sales: 0,
yesterday_returns: 0,
error: `custom sales/return columns hidden: salesPos=${salesPos} returnsPos=${returnsPos}`,
header_texts: headerTexts,
};
}
let totalSales = 0, totalReturns = 0; let totalSales = 0, totalReturns = 0;
const rowBreakdown = []; const rowBreakdown = [];
for (const tr of dataRows) { for (const tr of dataRows) {
const cells = Array.from(tr.querySelectorAll('td, th')); const cells = Array.from(tr.querySelectorAll('td, th'));
const salesCell = cells[salesCol]; const salesCell = cells[salesPos];
const returnsCell = cells[returnsCol]; const returnsCell = cells[returnsPos];
const head = (tr.innerText || tr.textContent || '').slice(0, 60); const head = (tr.innerText || tr.textContent || '').slice(0, 60);
if (!salesCell || !returnsCell) { if (!salesCell || !returnsCell) {
rowBreakdown.push({ rowBreakdown.push({
@@ -662,7 +731,7 @@ def _extract_with_text(table_frame: Any) -> tuple[dict[str, Any], str]:
ERP DOM often contains NBSP (\xa0) etc; Scrapling/Playwright on Windows gbk ERP DOM often contains NBSP (\xa0) etc; Scrapling/Playwright on Windows gbk
rejects those on return, so we use encodeURIComponent + tryUTF8 to make it ASCII-safe. rejects those on return, so we use encodeURIComponent + tryUTF8 to make it ASCII-safe.
按 MODE 选 idx 模式 (daily) 或 header 模式 (weekly). 按 MODE 选 idx 模式 (weekly) 或 header 模式 (daily / backfill)。
""" """
sales_key, returns_key = _sales_returns_idx() sales_key, returns_key = _sales_returns_idx()
@@ -673,17 +742,17 @@ def _extract_with_text(table_frame: Any) -> tuple[dict[str, Any], str]:
else: else:
js_args = {"salesHeader": sales_key, "returnsHeader": returns_key, "mode": "header"} js_args = {"salesHeader": sales_key, "returnsHeader": returns_key, "mode": "header"}
# 走 SPU 行 + 自定义列名匹配 + 求和 (因为 sumRow 在 ERP 系统里不含自定义段) # backfill 走 SPU 行 + 自定义列名匹配 + 求和 (因为 sumRow 在 ERP 系统里不含自定义段);
if sales_key != "__idx__45": # backfill 模式 # daily 的 昨日销量/昨日退货量 是原生列, sumRow 里就有, 走下面 payload 的
# mode='header' 分支 (表头名 -> x 坐标 -> sumRow 最近格), 不经过 _extract_by_header。
if MODE == "backfill":
header_result = _extract_by_header(table_frame, sales_key, returns_key) header_result = _extract_by_header(table_frame, sales_key, returns_key)
# _extract_by_header 返回 dict, 没有 body_text; wait_for_metrics 需要 erp_code in body_text 校验 # _extract_by_header 返回 dict, 没有 body_text; wait_for_metrics 需要 erp_code in body_text 校验
# 用 erp_code placeholder 骗校验 (它会查 erp_code in summary_text, 而 summary_text 不一定含 erp_code) # 用 erp_code placeholder 骗校验 (它会查 erp_code in summary_text, 而 summary_text 不一定含 erp_code)
# 但新逻辑在 row_breakdown 里已有 erp_code 关联的 SPU 编码 (10384xxx), 校验一定会过 # 但新逻辑在 row_breakdown 里已有 erp_code 关联的 SPU 编码 (10384xxx), 校验一定会过
return header_result, header_result.get("summary_text", "") or "PLACEHOLDER" return header_result, header_result.get("summary_text", "") or "PLACEHOLDER"
# 老 idx 模式 # weekly 老 idx 模式 + daily header 模式共用这段 payload (js_args 已按模式构造)
sales_idx = int(sales_key.removeprefix("__idx__"))
returns_idx = int(returns_key.removeprefix("__idx__"))
payload = table_frame.evaluate( payload = table_frame.evaluate(
""" """
(args) => { (args) => {
@@ -712,6 +781,9 @@ def _extract_with_text(table_frame: Any) -> tuple[dict[str, Any], str]:
} }
let salesCell = null, returnsCell = null; let salesCell = null, returnsCell = null;
// salesCol/returnsCol 供末尾 return 的 *_col_idx 引用, 必须在 if/else 外声明
// (2026-10-13 daily 切到 header 模式后首测即 ReferenceError 的教训)
let salesCol = -1, returnsCol = -1;
if (args.mode === 'idx') { if (args.mode === 'idx') {
const cells = Array.from(sumRow.children).map((cell) => { const cells = Array.from(sumRow.children).map((cell) => {
const box = cell.getBoundingClientRect(); const box = cell.getBoundingClientRect();
@@ -745,8 +817,8 @@ def _extract_with_text(table_frame: Any) -> tuple[dict[str, Any], str]:
} }
return -1; return -1;
}; };
const salesCol = findCol(args.salesHeader); salesCol = findCol(args.salesHeader);
const returnsCol = findCol(args.returnsHeader); returnsCol = findCol(args.returnsHeader);
if (salesCol < 0 || returnsCol < 0) { if (salesCol < 0 || returnsCol < 0) {
return { ok: false, yesterday_sales: 0, yesterday_returns: 0, return { ok: false, yesterday_sales: 0, yesterday_returns: 0,
error: `header not found: salesCol=${salesCol} returnsCol=${returnsCol}`, error: `header not found: salesCol=${salesCol} returnsCol=${returnsCol}`,
@@ -977,6 +1049,9 @@ def wait_for_metrics(page: Any, erp_code: str, timeout_seconds: int = 45,
# previous_url 可能在 _reload_allsku_iframe 后被重置为 None # previous_url 可能在 _reload_allsku_iframe 后被重置为 None
while True: while True:
# backfill 模式: 报表刷新会把自定义列的可见性重置回隐藏, 每轮恢复一次
if MODE == "backfill":
ensure_custom_columns_visible(page)
# 先检测"操作频繁"弹窗 (在主页 DOM, 不在 allsku iframe) # 先检测"操作频繁"弹窗 (在主页 DOM, 不在 allsku iframe)
if filter_frame is not None and freq_dialog_handled < MAX_FREQ_DIALOG: if filter_frame is not None and freq_dialog_handled < MAX_FREQ_DIALOG:
if _detect_and_close_freq_dialog(page): if _detect_and_close_freq_dialog(page):
@@ -1176,6 +1251,9 @@ def _run_one_code(page: Any, filter_frame: Any, platform: dict[str, Any], erp_co
""" """
clear_code(filter_frame) clear_code(filter_frame)
set_platform_and_code(filter_frame, platform["shops"], erp_code) set_platform_and_code(filter_frame, platform["shops"], erp_code)
if MODE == "backfill":
# 自定义销量/退货量列被模板重置隐藏时先恢复可见, 否则数据行不渲染对应格
ensure_custom_columns_visible(page)
prev_state = _get_allsku_frame_state(page) prev_state = _get_allsku_frame_state(page)
generate_report(filter_frame) generate_report(filter_frame)
return wait_for_metrics( return wait_for_metrics(
@@ -68,6 +68,9 @@
fact_market_rank_notifications fact_market_rank_notifications
写: market_rank_hermes_notification.py (Hermes 机器人通知结果;文件名兼容保留) 写: market_rank_hermes_notification.py (Hermes 机器人通知结果;文件名兼容保留)
读: 同一收件人同日通知去重 / 发送审计 读: 同一收件人同日通知去重 / 发送审计
erp_sales_theme_daily_metrics
写: collect_erp_sales_theme_daily.py (聚水潭「报表→销售主题分析→按日统计」Excel 导出入库)
读: 固定品牌+店铺口径的全渠道按日销售汇总查询
所有 upsert 都用 ON CONFLICT ... DO UPDATE, 重复跑覆盖不双写。 所有 upsert 都用 ON CONFLICT ... DO UPDATE, 重复跑覆盖不双写。
""" """
@@ -1908,3 +1911,160 @@ def upsert_market_rank_notification(conn, record: dict) -> int:
if not row: if not row:
raise RuntimeError("市场排行飞书通知归档未返回记录 id") raise RuntimeError("市场排行飞书通知归档未返回记录 id")
return int(row[0]) return int(row[0])
# ── erp_sales_theme_daily_metrics ────────────────────────────────────────────
def ensure_erp_sales_theme_daily_table(conn) -> None:
"""幂等创建聚水潭销售主题分析(按日)汇总表,不执行其他 schema 迁移。"""
sql = """
CREATE TABLE IF NOT EXISTS erp_sales_theme_daily_metrics (
id BIGSERIAL PRIMARY KEY,
stat_date DATE NOT NULL,
sales_order_count INTEGER NOT NULL DEFAULT 0,
sales_qty INTEGER NOT NULL DEFAULT 0,
shipped_qty INTEGER NOT NULL DEFAULT 0,
shipped_amount NUMERIC(18, 2) NOT NULL DEFAULT 0,
sales_amount NUMERIC(18, 2) NOT NULL DEFAULT 0,
return_order_count INTEGER NOT NULL DEFAULT 0,
return_qty INTEGER NOT NULL DEFAULT 0,
returned_qty INTEGER NOT NULL DEFAULT 0,
return_amount NUMERIC(18, 2) NOT NULL DEFAULT 0,
returned_amount NUMERIC(18, 2) NOT NULL DEFAULT 0,
net_sales_order_count INTEGER NOT NULL DEFAULT 0,
net_sales_qty INTEGER NOT NULL DEFAULT 0,
net_sales_amount NUMERIC(18, 2) NOT NULL DEFAULT 0,
freight_income NUMERIC(18, 2),
freight_expense NUMERIC(18, 2),
shipped_order_count INTEGER NOT NULL DEFAULT 0,
date_basis TEXT NOT NULL DEFAULT '发货时间',
brands TEXT[] NOT NULL DEFAULT '{}',
shops TEXT[] NOT NULL DEFAULT '{}',
source_file TEXT,
imported_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (stat_date)
);
CREATE INDEX IF NOT EXISTS erp_sales_theme_daily_metrics_date_idx
ON erp_sales_theme_daily_metrics (stat_date DESC);
COMMENT ON TABLE erp_sales_theme_daily_metrics IS
'聚水潭「报表→销售主题分析→按日统计」Excel 导出入库;固定品牌+店铺口径,每日一行幂等覆盖。';
COMMENT ON COLUMN erp_sales_theme_daily_metrics.date_basis IS
'报表时间口径,当前固定为发货时间(send_date)。';
COMMENT ON COLUMN erp_sales_theme_daily_metrics.brands IS
'本次查询勾选的商品品牌清单(筛选页 input[name=brand_name] 值)。';
COMMENT ON COLUMN erp_sales_theme_daily_metrics.shops IS
'本次查询勾选的销售渠道(店铺)名称清单。';
"""
with conn.cursor() as cur:
cur.execute(sql)
_SALES_THEME_INT_FIELDS = frozenset({
"sales_order_count",
"sales_qty",
"shipped_qty",
"return_order_count",
"return_qty",
"returned_qty",
"net_sales_order_count",
"net_sales_qty",
"shipped_order_count",
})
_SALES_THEME_COLUMNS = (
"sales_order_count",
"sales_qty",
"shipped_qty",
"shipped_amount",
"sales_amount",
"return_order_count",
"return_qty",
"returned_qty",
"return_amount",
"returned_amount",
"net_sales_amount",
"net_sales_order_count",
"net_sales_qty",
"shipped_order_count",
"freight_income",
"freight_expense",
)
# 表内 NOT NULL 的金额列;导出里无退货/无数据的日期这些单元格为空, 统一落 0
# (运费收入/支出两列允许 NULL, 与导出口径一致)。
_SALES_THEME_REQUIRED_AMOUNTS = frozenset({
"shipped_amount",
"sales_amount",
"return_amount",
"returned_amount",
"net_sales_amount",
})
def upsert_erp_sales_theme_daily_metrics(conn, records: Sequence[dict]) -> int:
"""幂等写入销售主题分析按日汇总,主键为统计日期。
records: [{'stat_date', 'sales_order_count', ..., 'brands', 'shops', 'source_file'}, ...]
数量字段缺省按 0 处理;brands/shops 写入前去重排序,保证重跑结果稳定。
"""
if not records:
return 0
dedup: dict[Any, dict] = {}
for record in records:
key = record["stat_date"]
if str(key).strip() == "":
raise ValueError("stat_date must not be empty")
dedup[key] = dict(record)
rows = []
for record in dedup.values():
row = [record["stat_date"]]
for field in _SALES_THEME_COLUMNS:
value = record.get(field)
if field in _SALES_THEME_INT_FIELDS:
row.append(int(value or 0))
elif field in _SALES_THEME_REQUIRED_AMOUNTS:
row.append(round(float(value), 2) if value is not None else 0)
else:
row.append(round(float(value), 2) if value is not None else None)
row.append(str(record.get("date_basis") or "发货时间"))
row.append(sorted({str(item).strip() for item in record.get("brands") or () if str(item).strip()}))
row.append(sorted({str(item).strip() for item in record.get("shops") or () if str(item).strip()}))
row.append(record.get("source_file"))
rows.append(tuple(row))
sql = """
INSERT INTO erp_sales_theme_daily_metrics
(stat_date, sales_order_count, sales_qty, shipped_qty,
shipped_amount, sales_amount,
return_order_count, return_qty, returned_qty,
return_amount, returned_amount, net_sales_amount,
net_sales_order_count, net_sales_qty,
shipped_order_count, freight_income, freight_expense,
date_basis, brands, shops, source_file)
VALUES %s
ON CONFLICT (stat_date) DO UPDATE SET
sales_order_count = EXCLUDED.sales_order_count,
sales_qty = EXCLUDED.sales_qty,
shipped_qty = EXCLUDED.shipped_qty,
shipped_amount = EXCLUDED.shipped_amount,
sales_amount = EXCLUDED.sales_amount,
return_order_count = EXCLUDED.return_order_count,
return_qty = EXCLUDED.return_qty,
returned_qty = EXCLUDED.returned_qty,
return_amount = EXCLUDED.return_amount,
returned_amount = EXCLUDED.returned_amount,
net_sales_order_count = EXCLUDED.net_sales_order_count,
net_sales_qty = EXCLUDED.net_sales_qty,
net_sales_amount = EXCLUDED.net_sales_amount,
freight_income = EXCLUDED.freight_income,
freight_expense = EXCLUDED.freight_expense,
shipped_order_count = EXCLUDED.shipped_order_count,
date_basis = EXCLUDED.date_basis,
brands = EXCLUDED.brands,
shops = EXCLUDED.shops,
source_file = EXCLUDED.source_file,
updated_at = NOW()
"""
with conn.cursor() as cur:
execute_values(cur, sql, rows)
return len(rows)
@@ -1273,3 +1273,37 @@ COMMENT ON TABLE fact_video_publications IS
'附件级视频发布幂等账本;点击发布后无明确回执必须进入 ambiguous,禁止自动重试。'; '附件级视频发布幂等账本;点击发布后无明确回执必须进入 ambiguous,禁止自动重试。';
COMMENT ON TABLE fact_video_publication_attempts IS COMMENT ON TABLE fact_video_publication_attempts IS
'视频发布每次尝试的标题、商品、话题、标签、提交时点与确认凭据。'; '视频发布每次尝试的标题、商品、话题、标签、提交时点与确认凭据。';
-- 聚水潭「报表→销售主题分析→按日统计」Excel 导出入库。
-- 口径:发货时间区间 + 固定品牌(光影行星/光影/gyxx) + 固定勾选店铺清单,
-- 采集脚本 collect_erp_sales_theme_daily.py;与 erp_all_shop_style_daily_metrics
-- (全店铺按款式) 分表,本表是全渠道合并口径的按日汇总。
CREATE TABLE IF NOT EXISTS erp_sales_theme_daily_metrics (
id BIGSERIAL PRIMARY KEY,
stat_date DATE NOT NULL,
sales_order_count INTEGER NOT NULL DEFAULT 0, -- 销售单数
sales_qty INTEGER NOT NULL DEFAULT 0, -- 销售数量
shipped_qty INTEGER NOT NULL DEFAULT 0, -- 实发数量
shipped_amount NUMERIC(18, 2) NOT NULL DEFAULT 0, -- 实发金额
sales_amount NUMERIC(18, 2) NOT NULL DEFAULT 0, -- 销售金额
return_order_count INTEGER NOT NULL DEFAULT 0, -- 退货单数
return_qty INTEGER NOT NULL DEFAULT 0, -- 退货数量
returned_qty INTEGER NOT NULL DEFAULT 0, -- 实退数量
return_amount NUMERIC(18, 2) NOT NULL DEFAULT 0, -- 退货金额
returned_amount NUMERIC(18, 2) NOT NULL DEFAULT 0, -- 实退金额
net_sales_order_count INTEGER NOT NULL DEFAULT 0, -- 净销售单数
net_sales_qty INTEGER NOT NULL DEFAULT 0, -- 净销量
net_sales_amount NUMERIC(18, 2) NOT NULL DEFAULT 0, -- 净销售额
freight_income NUMERIC(18, 2), -- 运费收入(可空)
freight_expense NUMERIC(18, 2), -- 运费支出(可空)
shipped_order_count INTEGER NOT NULL DEFAULT 0, -- 实发单数
date_basis TEXT NOT NULL DEFAULT '发货时间',
brands TEXT[] NOT NULL DEFAULT '{}',
shops TEXT[] NOT NULL DEFAULT '{}',
source_file TEXT,
imported_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (stat_date)
);
CREATE INDEX IF NOT EXISTS erp_sales_theme_daily_metrics_date_idx
ON erp_sales_theme_daily_metrics (stat_date DESC);
@@ -29,15 +29,19 @@ from runtime_paths import TMP_ROOT
from gyxx_flow.adapters.browser import BrowserCookieStore from gyxx_flow.adapters.browser import BrowserCookieStore
from gyxx_flow.adapters.scrapling import ScraplingBrowser, is_browser_timeout_error from gyxx_flow.adapters.scrapling import ScraplingBrowser, is_browser_timeout_error
from gyxx_flow.core.config import Settings
from gyxx_flow.core.locks import LockManager, ResourceBusyError from gyxx_flow.core.locks import LockManager, ResourceBusyError
_CONFIG_NAME = "tmall-baibu.json" _CONFIG_NAME = "tmall-baibu.json"
_LOGIN_PAGE_URL = "https://login.taobao.com/member/login.jhtml"
_EXPORT_TIMEOUT_SECONDS = 240 _EXPORT_TIMEOUT_SECONDS = 240
_EXPORT_MAX_ATTEMPTS = 3 _EXPORT_MAX_ATTEMPTS = 3
_EXPORT_RETRY_DELAYS_SECONDS = (5, 15) _EXPORT_RETRY_DELAYS_SECONDS = (5, 15)
_PAGE_TIMEOUT_MS = 60_000 _PAGE_TIMEOUT_MS = 60_000
_IMPORT_TIMEOUT_SECONDS = 300 _IMPORT_TIMEOUT_SECONDS = 300
_IMPORT_ACCEPTED_AFTER_PROCESSING_SECONDS = 120
_STATUS_SETTLE_TIMEOUT_SECONDS = 5 _STATUS_SETTLE_TIMEOUT_SECONDS = 5
_ACCOUNT_LOCK_TIMEOUT_SECONDS = 120
_RESULT_REFRESH_INTERVAL_SECONDS = 8 _RESULT_REFRESH_INTERVAL_SECONDS = 8
_DRAFT_ACTION_TIMEOUT_SECONDS = 30 _DRAFT_ACTION_TIMEOUT_SECONDS = 30
_DRAFT_NAVIGATION_TIMEOUT_SECONDS = 15 _DRAFT_NAVIGATION_TIMEOUT_SECONDS = 15
@@ -103,6 +107,7 @@ _LOGIN_SECOND_FACTOR_MARKERS = (
"安全验证", "安全验证",
"二次验证", "二次验证",
"验证码", "验证码",
"滑块",
) )
@@ -805,6 +810,41 @@ def _find_password_login_form(
return locate() return locate()
def _is_login_agreement_visible(page: Any) -> bool:
"""Return whether Tmall's 服务协议及隐私保护 popup is blocking the login."""
for context in _page_contexts(page):
get_by_role = getattr(context, "get_by_role", None)
if not callable(get_by_role):
continue
try:
dialogs = get_by_role("dialog")
for index in range(min(dialogs.count(), 8)):
dialog = dialogs.nth(index)
try:
if not dialog.is_visible():
continue
text = _normalized_page_text(dialog.inner_text(timeout=2_000))
except Exception:
continue
if "服务协议" in text or "隐私" in text:
return True
except Exception:
continue
return False
def _click_login_agreement_accept(page: Any) -> bool:
for context in _page_contexts(page):
for candidate in _visible_text_candidates(context, "同意"):
try:
candidate.click(timeout=8_000)
return True
except Exception:
continue
return False
def _login_if_needed( def _login_if_needed(
entry: BaibuEntry, entry: BaibuEntry,
page: Any, page: Any,
@@ -817,6 +857,11 @@ def _login_if_needed(
return False return False
login_url = _is_login_url(str(getattr(page, "url", ""))) login_url = _is_login_url(str(getattr(page, "url", "")))
form = _find_password_login_form(page, use_password_form=False) form = _find_password_login_form(page, use_password_form=False)
if form is None and not login_url:
# A redirect can land on a scanner-tab seller login page (qianniu)
# whose URL carries no login marker; probe the password tab before
# deciding there is nothing to log in to.
form = _find_password_login_form(page, use_password_form=True)
if not login_url and form is None: if not login_url and form is None:
return False return False
@@ -852,10 +897,92 @@ def _login_if_needed(
raise BaibuApplyError( raise BaibuApplyError(
f"{entry.name} 自动填写账号密码并提交登录失败({type(exc).__name__})" f"{entry.name} 自动填写账号密码并提交登录失败({type(exc).__name__})"
) from exc ) from exc
# The first password submit can pop the 服务协议及隐私保护 confirmation
# which swallows the submit; accept it and submit once more so the login
# actually runs. Both popups and form re-renders are tolerated.
for _attempt in range(2):
wait_for_timeout = getattr(page, "wait_for_timeout", None)
if callable(wait_for_timeout):
wait_for_timeout(2_000)
else:
time.sleep(2)
if not _is_login_agreement_visible(page):
break
if not _click_login_agreement_accept(page):
break
print(f"[百补] {entry.name} 已确认登录服务协议弹窗,重新提交登录")
refreshed = _find_password_login_form(page, use_password_form=True)
if refreshed is None:
break
_context, username, secret_input, submit = refreshed
try:
username.fill(credentials.username)
secret_input.fill(credentials.password)
submit.click(timeout=15_000)
except Exception as exc:
raise BaibuApplyError(
f"{entry.name} 协议确认后重新提交登录失败({type(exc).__name__})"
) from exc
print(f"[百补] {entry.name} 已提交密码登录,等待店铺身份确认") print(f"[百补] {entry.name} 已提交密码登录,等待店铺身份确认")
return True return True
def _goto_tolerant(page: Any, url: str) -> None:
"""Navigate, tolerating an aborted load when a redirect takes over.
An unauthenticated apply URL is answered with a redirect chain that can
abort the original navigation from inside the browser; treat that as the
navigation landing wherever the chain pointed and let it settle.
"""
try:
page.goto(url, wait_until="domcontentloaded", timeout=_PAGE_TIMEOUT_MS)
except Exception as exc:
if "ERR_ABORTED" not in str(exc):
raise
wait_for_timeout = getattr(page, "wait_for_timeout", None)
if callable(wait_for_timeout):
wait_for_timeout(_LOGIN_FORM_SETTLE_MILLISECONDS)
else:
time.sleep(_LOGIN_FORM_SETTLE_MILLISECONDS / 1000)
def _explicit_password_login(
entry: BaibuEntry,
page: Any,
*,
manual_login: bool,
) -> None:
"""Drive the Tmall password form when the apply page hid its login state.
The myseller/qn apply SPA can sit on an SSO interstitial that never shows
a login URL, or render the cached shop name on an expired session. In
both cases ``_login_if_needed`` sees nothing to do. Re-entering the apply
URL redirects to the seller login page (qianniu) which carries the right
merchant context; only when that shows no form fall back to the plain
login page. Login there, return to the apply URL, and confirm identity.
"""
print(f"[百补] {entry.name} 显式重新进入报名页并走密码登录")
_goto_tolerant(page, entry.apply_url)
wait_for_timeout = getattr(page, "wait_for_timeout", None)
if callable(wait_for_timeout):
wait_for_timeout(_LOGIN_FORM_SETTLE_MILLISECONDS * 3)
else:
time.sleep(_LOGIN_FORM_SETTLE_MILLISECONDS / 1000 * 3)
login_attempted = _login_if_needed(entry, page, manual_login=manual_login)
if not login_attempted:
_goto_tolerant(page, _LOGIN_PAGE_URL)
login_attempted = _login_if_needed(entry, page, manual_login=manual_login)
_goto_tolerant(page, entry.apply_url)
_wait_for_authenticated(
entry,
page,
login_attempted=login_attempted,
manual_login=manual_login,
)
def _click_text(page: Any, labels: Iterable[str]) -> bool: def _click_text(page: Any, labels: Iterable[str]) -> bool:
contexts = _page_contexts(page) contexts = _page_contexts(page)
for label in labels: for label in labels:
@@ -1040,7 +1167,10 @@ def _open_batch_import(
f"{entry.name} 页面已有批量导入操作正在处理,已停止,避免重复提交" f"{entry.name} 页面已有批量导入操作正在处理,已停止,避免重复提交"
) )
time.sleep(0.5) time.sleep(0.5)
raise BaibuApplyError(f"{entry.name} 页面没有找到可用的「商品批量导入」入口") raise BaibuApplyError(
f"{entry.name} 页面没有找到可用的「商品批量导入」入口"
"(活动可能已报名/已结束或会话无效,请人工打开报名页确认)"
)
def _wait_import_result( def _wait_import_result(
@@ -1062,6 +1192,7 @@ def _wait_import_result(
last_processing_log = 0.0 last_processing_log = 0.0
last_refresh = 0.0 last_refresh = 0.0
observed_new_operation = False observed_new_operation = False
first_new_operation_at: float | None = None
while time.monotonic() < deadline: while time.monotonic() < deadline:
text = _page_text(page) text = _page_text(page)
operation = _latest_batch_import_status(text) operation = _latest_batch_import_status(text)
@@ -1070,6 +1201,8 @@ def _wait_import_result(
or operation.signature != previous_operation.signature or operation.signature != previous_operation.signature
): ):
observed_new_operation = True observed_new_operation = True
if first_new_operation_at is None:
first_new_operation_at = time.monotonic()
if operation.state == "success": if operation.state == "success":
print( print(
f"[百补] {entry.name} 页面已完成导入处理," f"[百补] {entry.name} 页面已完成导入处理,"
@@ -1082,6 +1215,21 @@ def _wait_import_result(
f"忽略行结果并按导入提交成功:{operation.detail}" f"忽略行结果并按导入提交成功:{operation.detail}"
) )
return return
if (
observed_new_operation
and first_new_operation_at is not None
and time.monotonic() - first_new_operation_at
>= _IMPORT_ACCEPTED_AFTER_PROCESSING_SECONDS
):
# Tmall has already acknowledged the batch ("执行了商品批量导入
# 操作,正在处理中"); the submission itself is the business
# effect, so a slow per-row summary must not fail the run.
print(
f"[百补] {entry.name} 天猫已受理批量导入,"
f"{_IMPORT_ACCEPTED_AFTER_PROCESSING_SECONDS} 秒内未返回行结果,"
"按导入提交成功,行结果以报名页面为准"
)
return
if observed_new_operation and any(marker in text for marker in _SUCCESS_MARKERS): if observed_new_operation and any(marker in text for marker in _SUCCESS_MARKERS):
print(f"[百补] {entry.name} 页面已确认导入提交,忽略行结果") print(f"[百补] {entry.name} 页面已确认导入提交,忽略行结果")
return return
@@ -1284,9 +1432,20 @@ def _complete_draft_products(page: Any, entry: BaibuEntry) -> None:
if not entry.complete_drafts_after_import: if not entry.complete_drafts_after_import:
return return
if not _click_draft_tab(page): # Right after the import acknowledgement the SPA may still be hydrating;
count = _draft_count(_page_text(page)) # the draft tab, the draft count and the per-row action rarely all appear
if count is None and not _has_visible_text(page, ("完善商品",)): # on the first probe, so retry briefly before declaring the page broken.
deadline = time.monotonic() + _DRAFT_NAVIGATION_TIMEOUT_SECONDS
while time.monotonic() < deadline:
if _click_draft_tab(page):
break
text = _page_text(page)
if _draft_count(text) is not None or _has_visible_text(page, ("完善商品",)):
break
if any(marker in text for marker in _DRAFT_EMPTY_MARKERS):
break
time.sleep(0.5)
else:
raise BaibuApplyError(f"{entry.name} 未找到草稿页入口") raise BaibuApplyError(f"{entry.name} 未找到草稿页入口")
draft_count, action_count = _wait_for_draft_state(page, entry) draft_count, action_count = _wait_for_draft_state(page, entry)
@@ -1439,6 +1598,8 @@ def _has_account_cookie(
def _persist_account_state( def _persist_account_state(
browser: ScraplingBrowser, browser: ScraplingBrowser,
entry: BaibuEntry, entry: BaibuEntry,
*,
session_lock_held: bool = False,
) -> bool: ) -> bool:
"""Publish a verified login to the account vault without leaking cookies.""" """Publish a verified login to the account vault without leaking cookies."""
@@ -1463,8 +1624,12 @@ def _persist_account_state(
) )
cookies = _account_cookies(cookies, required_domains) cookies = _account_cookies(cookies, required_domains)
if not _has_account_cookie(cookies, required_names): if not _has_account_cookie(cookies, required_names):
names = sorted({str(item.get("name", "")) for item in cookies})
preview = "、".join(names[:12]) if names else "无"
raise BaibuApplyError( raise BaibuApplyError(
f"{entry.name} 登录后未获得可用的账号 cookie,未更新账号登录态" f"{entry.name} 登录后未获得可用的账号 cookie"
f"(域内 cookie:{preview};要求含:{'、'.join(required_names)}),"
"未更新账号登录态"
) )
raw_state = context.storage_state() raw_state = context.storage_state()
@@ -1472,23 +1637,33 @@ def _persist_account_state(
filtered_state = _account_storage_state(state, cookies, required_domains) filtered_state = _account_storage_state(state, cookies, required_domains)
cookie_file, storage_state_file = paths cookie_file, storage_state_file = paths
store = BrowserCookieStore(cookie_file, storage_state_file) store = BrowserCookieStore(cookie_file, storage_state_file)
lock_root = cookie_file.parent.parent / "locks"
def _save() -> None:
try:
store.save_cookies(cookies)
store.save_storage_state(filtered_state)
except (OSError, ValueError, TypeError) as exc:
raise BaibuApplyError(
f"{entry.name} 保存账号登录态失败({type(exc).__name__})"
) from exc
if session_lock_held:
# The whole browser session already holds the scheduler-side
# browser-account lock; re-acquiring here would self-deadlock.
_save()
else:
lock_root = Settings.from_env().data_root / "state" / "locks"
try: try:
with LockManager(lock_root).acquire( with LockManager(lock_root).acquire(
f"browser-account:{entry.account_id}", f"browser-account:{entry.account_id}",
owner=f"tmall-baibu:{entry.entry_id}", owner=f"tmall-baibu:{entry.entry_id}",
timeout_seconds=0, timeout_seconds=0,
): ):
store.save_cookies(cookies) _save()
store.save_storage_state(filtered_state)
except ResourceBusyError as exc: except ResourceBusyError as exc:
raise BaibuApplyError( raise BaibuApplyError(
f"{entry.name} 账号登录态正在被其他进程更新,未覆盖账号登录态" f"{entry.name} 账号登录态正在被其他进程更新,未覆盖账号登录态"
) from exc ) from exc
except (OSError, ValueError, TypeError) as exc:
raise BaibuApplyError(
f"{entry.name} 保存账号登录态失败({type(exc).__name__})"
) from exc
print(f"[百补] 已将 {entry.account_id} 的已验证登录态同步到账号 vault") print(f"[百补] 已将 {entry.account_id} 的已验证登录态同步到账号 vault")
return True return True
@@ -1498,6 +1673,33 @@ def _apply_entry(
template: Path, template: Path,
*, *,
manual_login: bool = False, manual_login: bool = False,
) -> None:
# Hold the scheduler-side account browser lock for the whole session.
# The resident keepalive heartbeats the same profile on an interval and
# its timeout-0 acquire skips busy accounts, so this lock is what stops
# the heartbeat from killing the import mid-run (TargetClosedError /
# silent page death). Members must use the same lock directory as the
# scheduler (Settings.data_root/state/locks), not one derived from the
# vault paths.
locks = LockManager(Settings.from_env().data_root / "state" / "locks")
try:
with locks.acquire(
f"browser-account:{entry.account_id}",
owner=f"tmall-baibu:{entry.entry_id}",
timeout_seconds=_ACCOUNT_LOCK_TIMEOUT_SECONDS,
):
_apply_entry_locked(entry, template, manual_login=manual_login)
except ResourceBusyError as exc:
raise BaibuApplyError(
f"{entry.name} 账号浏览器正被保活/登录占用,请稍后重试"
) from exc
def _apply_entry_locked(
entry: BaibuEntry,
template: Path,
*,
manual_login: bool = False,
) -> None: ) -> None:
browser = _start_browser() browser = _start_browser()
try: try:
@@ -1516,14 +1718,36 @@ def _apply_entry(
page, page,
manual_login=manual_login, manual_login=manual_login,
) )
try:
_wait_for_authenticated( _wait_for_authenticated(
entry, entry,
page, page,
login_attempted=login_attempted, login_attempted=login_attempted,
manual_login=manual_login, manual_login=manual_login,
) )
except BaibuApplyError:
if login_attempted or manual_login:
raise
# The apply page may sit on an SSO interstitial without a login
# URL, so the passive check above never triggers the credentials;
# fall back to the explicit password login once.
_explicit_password_login(entry, page, manual_login=manual_login)
login_attempted = True
browser.persist_bound_state() browser.persist_bound_state()
_persist_account_state(browser, entry) try:
_persist_account_state(browser, entry, session_lock_held=True)
except BaibuApplyError as exc:
# A failed vault refresh can also mean the shop name was rendered
# from cache on top of a guest session; retry once through the
# explicit login before giving up on the vault refresh. The
# refresh only caches this session for later runs and must never
# block the import submission itself.
print(f"[WARN] {exc};尝试显式重新登录后重试", file=sys.stderr)
try:
_explicit_password_login(entry, page, manual_login=manual_login)
_persist_account_state(browser, entry, session_lock_held=True)
except BaibuApplyError as retry_exc:
print(f"[WARN] {retry_exc};本次继续执行报名导入", file=sys.stderr)
print(f"[百补] 已确认账号 {entry.account_id},打开 {entry.name} 报名入口") print(f"[百补] 已确认账号 {entry.account_id},打开 {entry.name} 报名入口")
previous_operation = _open_batch_import(page, entry) previous_operation = _open_batch_import(page, entry)
time.sleep(1) time.sleep(1)
+117
View File
@@ -0,0 +1,117 @@
# 运维审计库(ops_* 表)
工作流执行历史与登录 cookie 遥测的 PostgreSQL 落库。本地 `var/runs/**/run.json`
仍是权威源,这里做的是**跨机器可查、可长期留存的关系型镜像**。
## 为什么需要
- 本地 journal 只在部署机磁盘上:换机、扩容、清理后历史就断了。
- `var/logs` 是文本,做不了「近 30 天哪些工作流失败最多」这类查询。
- cookie 状态原先只存在 `keepalive_status.json`(只有最新值),无法回答
「登录态到底能活几天」。
## 表与视图
| 对象 | 粒度 | 用途 |
|---|---|---|
| `ops_workflow_runs` | 一次运行一行 | 状态、模式、耗时、步骤计数、错误、journal 路径、主机 |
| `ops_workflow_run_steps` | 一次尝试一行 | 含重试;`exit_code`、错误、耗时 |
| `ops_account_cookie_events` | 一次观测一行 | 事件流;只存指纹,不存 cookie 明文 |
| `ops_account_cookie_lifetimes`(视图) | 一份 cookie 一行 | 从首次出现到失效的寿命 |
| `ops_account_cookie_stats`(视图) | 一个账号一行 | 平均 / 中位 / 最短 / 最长寿命 |
## 自动建表
首次写入时执行 `SCHEMA_SQL`(幂等 DDL),无需手工迁移。也可以用:
```bash
uv run gyxx audit schema > 001_ops_audit.sql # 导出 DDL 交 DBA 预建
psql "$GYXX_POSTGRES_DSN" -f 001_ops_audit.sql
```
## 查询
```bash
uv run gyxx audit runs --workflow content.metrics.self_operated.daily --limit 20
uv run gyxx audit runs --status failed --limit 50
uv run gyxx audit cookies --account douyin-shop --limit 50
uv run gyxx audit cookie-stats # 各账号失效规律
uv run gyxx audit cookie-stats --account douyin-shop # 单账号每次的寿命明细
```
都支持 `--json`,方便接到面板或告警脚本。
直接 SQL 也一样:
```sql
-- 最近 7 天失败最多的 10 个工作流
SELECT workflow_id, COUNT(*) AS failures
FROM ops_workflow_runs
WHERE status = 'failed' AND started_at > NOW() - INTERVAL '7 days'
GROUP BY workflow_id ORDER BY failures DESC LIMIT 10;
-- 每次失败卡在哪个步骤
SELECT r.workflow_id, s.step_id, COUNT(*) AS n
FROM ops_workflow_run_steps s
JOIN ops_workflow_runs r USING (run_id)
WHERE s.status = 'failed' AND r.started_at > NOW() - INTERVAL '7 days'
GROUP BY 1, 2 ORDER BY n DESC;
-- 哪个账号的登录态最短命(预警阈值参考)
SELECT account_id, sessions, expired_sessions,
avg_lifetime_seconds / 86400.0 AS avg_days,
min_lifetime_seconds / 86400.0 AS min_days,
last_failed_at
FROM ops_account_cookie_stats ORDER BY min_lifetime_seconds NULLS LAST;
```
## 怎么读「失效规律」
一次 cookie 代由 `fingerprint` 标识(需求域名/名称/值哈希排序后取摘要,与文件顺序无关):
- `first_healthy_at` → `first_failed_at` = `lifetime_seconds`:**实际可用时长**
- `healthy_to_failure_seconds`:最后一次探活成功到发现失效,反映**探测精度**
- `declared_expires_at`:cookie 自带的最早过期时间
把 `lifetime_seconds` 和 `declared_expires_at` 对比,能区分三种情况:
1. 实际寿命接近声明寿命 → 平台按声明失效,可按 `declared_expires_at` 提前预警;
2. 实际远短于声明 → 服务端提前吊销(常见于多端登录、风控);
3. `declared_expires_at` 为空(session cookie)→ 只能靠观测,`min_lifetime_seconds`
就是安全刷新周期的上界。
`fingerprint` 变化表示换了一份 cookie(重新登录),不算失效,也不会把两段寿命混在一起。
## 采集点
| 来源 | 触发 | `probe_source` |
|---|---|---|
| `CookieGate.check` | 每次工作流运行前的门禁判定 | `gate` |
| `AccountKeepaliveManager` | 常驻调度器每 `keepalive_interval_seconds`(默认 600s) | `keepalive` |
| `RunIndex.index_journal` | 每次运行开始/结束/异常 | — |
keepalive 是主要的精度来源:默认 10 分钟一次,同类状态一小时内只落一行,
状态翻转立即落行,所以「首次失效时间」误差不超过一个探活周期。
## 容错
审计**永不阻断业务**:
- 数据库不可达 / 缺 DSN / 建表失败 → 写入返回 `False`,调用方继续执行;
- 连续失败 3 次后熔断冷却(默认 300s),避免每个工作流都卡在连接超时;
- 失败记录写到 `<GYXX_DATA_ROOT>/logs/ops-audit/audit-failures.log`。
## 开关
| 变量 | 默认 | 说明 |
|---|---|---|
| `GYXX_OPS_AUDIT` | 开 | 设 `off` 关闭全部审计写入 |
| `GYXX_OPS_AUDIT_HOST` | 主机名 | 多机部署时写入手工标识 |
| `GYXX_OPS_AUDIT_FAILURE_COOLDOWN` | 300 | 熔断冷却秒数 |
## 不存什么
- **cookie 值**:只存哈希指纹;排查具体 cookie 请走 vault 文件。
- **日志正文**:`ops_workflow_runs` 只存 `journal_path` 指向本地文件;
完整堆栈仍在 `var/logs/console-restarts/`。
- **凭据**:DSN 只从 `GYXX_POSTGRES_DSN`(或 `PG_*`)读,不落表。
+5 -1
View File
@@ -1,5 +1,6 @@
"""Operational run indexing and durable outbox services.""" """Operational run indexing, audit telemetry and durable outbox services."""
from .audit import AuditStore, CookieSnapshot, summarize_cookies
from .effects import ( from .effects import (
EffectAlreadyApplied, EffectAlreadyApplied,
EffectClaim, EffectClaim,
@@ -10,6 +11,8 @@ from .index import RunIndex, RunRecord
from .outbox import IdempotencyConflict, Outbox, OutboxMessage from .outbox import IdempotencyConflict, Outbox, OutboxMessage
__all__ = [ __all__ = [
"AuditStore",
"CookieSnapshot",
"IdempotencyConflict", "IdempotencyConflict",
"EffectAlreadyApplied", "EffectAlreadyApplied",
"EffectClaim", "EffectClaim",
@@ -19,4 +22,5 @@ __all__ = [
"OutboxMessage", "OutboxMessage",
"RunIndex", "RunIndex",
"RunRecord", "RunRecord",
"summarize_cookies",
] ]
+901
View File
@@ -0,0 +1,901 @@
"""PostgreSQL-backed operational audit: workflow run history and cookie telemetry.
设计原则
--------
1. **可观测性不拖累业务**:审计写入永远不向调用方抛异常。数据库不可达时
只记录一条 failure 日志并熔断冷却,工作流照常执行。
2. **本地 journal 仍是权威源**:本模块是镜像/汇总,供跨机器查询与长期留存;
`var/runs/**/run.json` 不会被替代。
3. **不落明文凭据**:cookie 只存指纹(域名+名称+值哈希),值本身永不入库。
环境开关
--------
``GYXX_OPS_AUDIT=off`` 关闭审计(默认开启)
``GYXX_OPS_AUDIT_HOST`` 覆盖主机标识(默认 socket.gethostname())
``GYXX_OPS_AUDIT_FAILURE_COOLDOWN`` 熔断冷却秒数(默认 300)
``GYXX_POSTGRES_DSN`` 与动态配置共用同一 DSN
"""
from __future__ import annotations
import hashlib
import json
import os
import socket
from contextlib import contextmanager
from dataclasses import dataclass
from datetime import date, datetime, timezone
from pathlib import Path
from typing import Any, Iterator, Mapping, Sequence
SCHEMA_VERSION = 1
OPS_AUDIT_ENV = "GYXX_OPS_AUDIT"
OPS_AUDIT_HOST_ENV = "GYXX_OPS_AUDIT_HOST"
OPS_AUDIT_COOLDOWN_ENV = "GYXX_OPS_AUDIT_FAILURE_COOLDOWN"
_DEFAULT_COOLDOWN_SECONDS = 300
_MAX_CONSECUTIVE_FAILURES = 3
_MAX_TEXT_CHARS = 4000
_SCHEMA_LOCK_KEY = "gyxx:ops-audit:migration"
HEALTHY_STATES = frozenset({"healthy"})
FAILED_STATES = frozenset({"expired", "not_logged_in"})
_COOKIE_STATES = frozenset(
{"healthy", "expired", "not_logged_in", "probe_error", "busy", "disabled"}
)
SCHEMA_SQL = """
CREATE TABLE IF NOT EXISTS ops_workflow_runs (
run_id TEXT PRIMARY KEY,
workflow_id TEXT NOT NULL,
business_date DATE NOT NULL,
mode TEXT NOT NULL DEFAULT 'unknown',
shadow BOOLEAN NOT NULL DEFAULT FALSE,
status TEXT NOT NULL,
started_at TIMESTAMPTZ,
ended_at TIMESTAMPTZ,
duration_seconds INTEGER,
error TEXT,
step_total INTEGER NOT NULL DEFAULT 0,
step_success INTEGER NOT NULL DEFAULT 0,
step_failed INTEGER NOT NULL DEFAULT 0,
step_skipped INTEGER NOT NULL DEFAULT 0,
step_running INTEGER NOT NULL DEFAULT 0,
journal_path TEXT NOT NULL DEFAULT '',
host TEXT NOT NULL DEFAULT '',
indexed_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS ops_workflow_runs_workflow_idx
ON ops_workflow_runs (workflow_id, started_at DESC);
CREATE INDEX IF NOT EXISTS ops_workflow_runs_status_idx
ON ops_workflow_runs (status, started_at DESC);
CREATE INDEX IF NOT EXISTS ops_workflow_runs_business_date_idx
ON ops_workflow_runs (business_date, workflow_id);
CREATE TABLE IF NOT EXISTS ops_workflow_run_steps (
run_id TEXT NOT NULL
REFERENCES ops_workflow_runs (run_id) ON DELETE CASCADE,
step_id TEXT NOT NULL,
attempt INTEGER NOT NULL DEFAULT 1,
status TEXT NOT NULL DEFAULT '',
exit_code INTEGER,
started_at TIMESTAMPTZ,
ended_at TIMESTAMPTZ,
duration_seconds INTEGER,
error TEXT,
PRIMARY KEY (run_id, step_id, attempt)
);
CREATE INDEX IF NOT EXISTS ops_workflow_run_steps_step_idx
ON ops_workflow_run_steps (step_id, status);
CREATE TABLE IF NOT EXISTS ops_account_cookie_events (
id BIGSERIAL PRIMARY KEY,
account_id TEXT NOT NULL,
observed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
state TEXT NOT NULL,
probe_status TEXT NOT NULL DEFAULT '',
probe_source TEXT NOT NULL DEFAULT '',
cookie_count INTEGER,
required_count INTEGER,
fingerprint TEXT NOT NULL DEFAULT '',
declared_expires_at TIMESTAMPTZ,
detail TEXT NOT NULL DEFAULT '',
workflow_id TEXT NOT NULL DEFAULT '',
run_id TEXT NOT NULL DEFAULT '',
host TEXT NOT NULL DEFAULT ''
);
CREATE INDEX IF NOT EXISTS ops_account_cookie_events_account_idx
ON ops_account_cookie_events (account_id, observed_at DESC);
CREATE INDEX IF NOT EXISTS ops_account_cookie_events_state_idx
ON ops_account_cookie_events (state, observed_at DESC);
CREATE INDEX IF NOT EXISTS ops_account_cookie_events_fingerprint_idx
ON ops_account_cookie_events (account_id, fingerprint);
-- 一段「cookie 代」的生命周期:同一 fingerprint 从首次出现到失效。
-- first_failed_at 只取最后一次 healthy 之后的失败,避免恢复后再次失败时口径前移。
CREATE OR REPLACE VIEW ops_account_cookie_lifetimes AS
WITH per_fingerprint AS (
SELECT
account_id,
fingerprint,
MIN(observed_at) AS first_seen_at,
MIN(observed_at) FILTER (WHERE state = 'healthy') AS first_healthy_at,
MAX(observed_at) FILTER (WHERE state = 'healthy') AS last_healthy_at,
MAX(declared_expires_at) AS declared_expires_at
FROM ops_account_cookie_events
WHERE fingerprint <> ''
GROUP BY account_id, fingerprint
)
SELECT
p.account_id,
p.fingerprint,
p.first_seen_at,
p.first_healthy_at,
p.last_healthy_at,
p.declared_expires_at,
f.first_failed_at,
(f.first_failed_at IS NOT NULL) AS expired,
CASE
WHEN p.first_healthy_at IS NOT NULL AND p.last_healthy_at IS NOT NULL
THEN EXTRACT(EPOCH FROM (p.last_healthy_at - p.first_healthy_at))::BIGINT
END AS healthy_span_seconds,
CASE
WHEN p.first_healthy_at IS NOT NULL AND f.first_failed_at IS NOT NULL
THEN EXTRACT(EPOCH FROM (f.first_failed_at - p.first_healthy_at))::BIGINT
END AS lifetime_seconds,
CASE
WHEN p.last_healthy_at IS NOT NULL AND f.first_failed_at IS NOT NULL
THEN EXTRACT(EPOCH FROM (f.first_failed_at - p.last_healthy_at))::BIGINT
END AS healthy_to_failure_seconds
FROM per_fingerprint p
LEFT JOIN LATERAL (
SELECT MIN(e.observed_at) AS first_failed_at
FROM ops_account_cookie_events e
WHERE e.account_id = p.account_id
AND e.fingerprint = p.fingerprint
AND e.state IN ('expired', 'not_logged_in')
AND (p.last_healthy_at IS NULL OR e.observed_at > p.last_healthy_at)
) f ON TRUE;
-- 按账号汇总的失效规律:平均/中位/最短/最长寿命,直接支撑「多久失效一次」。
CREATE OR REPLACE VIEW ops_account_cookie_stats AS
SELECT
account_id,
COUNT(*) AS sessions,
COUNT(*) FILTER (WHERE expired) AS expired_sessions,
AVG(lifetime_seconds)::BIGINT AS avg_lifetime_seconds,
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY lifetime_seconds)::BIGINT
AS median_lifetime_seconds,
MIN(lifetime_seconds) AS min_lifetime_seconds,
MAX(lifetime_seconds) AS max_lifetime_seconds,
MAX(first_failed_at) AS last_failed_at,
MAX(last_healthy_at) AS last_healthy_at
FROM ops_account_cookie_lifetimes
WHERE lifetime_seconds IS NOT NULL
GROUP BY account_id;
CREATE TABLE IF NOT EXISTS ops_audit_schema (
schema_version INTEGER PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
"""
# ----------------------------------------------------------------------
# 纯函数:cookie 快照
# ----------------------------------------------------------------------
@dataclass(frozen=True, slots=True)
class CookieSnapshot:
"""一次静态检查得到的 cookie 画像,不含任何明文值。"""
cookie_count: int
required_count: int
fingerprint: str
declared_expires_at: datetime | None
def to_dict(self) -> dict[str, Any]:
return {
"cookie_count": self.cookie_count,
"required_count": self.required_count,
"fingerprint": self.fingerprint,
"declared_expires_at": (
self.declared_expires_at.isoformat()
if self.declared_expires_at is not None
else None
),
}
def _domain_matches(domain: str, required_domains: Sequence[str]) -> bool:
normalized = str(domain or "").lstrip(".").casefold()
if not normalized:
return False
return any(
normalized == required.casefold() or normalized.endswith("." + required.casefold())
for required in required_domains
)
def _expires_at(cookie: Mapping[str, Any]) -> datetime | None:
raw = cookie.get("expires")
if raw is None or isinstance(raw, bool):
return None
try:
value = float(raw)
except (TypeError, ValueError):
return None
# Playwright 用 -1 表示 session cookie;0 与负数都视为「不声明」。
if value <= 0:
return None
try:
return datetime.fromtimestamp(value, tz=timezone.utc)
except (OverflowError, OSError, ValueError):
return None
def summarize_cookies(
cookies: Sequence[Mapping[str, Any]],
*,
required_domains: Sequence[str] = (),
required_names: Sequence[str] = (),
) -> CookieSnapshot:
"""Build a value-free snapshot: counts, fingerprint and earliest declared expiry.
指纹只看「必需 cookie」的域名/名称/值哈希,因此登录态真正换了一份 cookie
时指纹会变,而无关 cookie 的增删不会造成抖动。
"""
required_domains = tuple(item for item in required_domains if item)
required_names = tuple(item for item in required_names if item)
components: list[str] = []
required_count = 0
earliest: datetime | None = None
for cookie in cookies:
if not isinstance(cookie, Mapping):
continue
name = str(cookie.get("name", ""))
domain = str(cookie.get("domain", ""))
is_required = bool(required_domains) and _domain_matches(domain, required_domains)
if required_names:
is_required = is_required and name in required_names
elif not required_domains:
is_required = True
if not is_required:
continue
required_count += 1
value = str(cookie.get("value", ""))
value_hash = hashlib.sha256(value.encode("utf-8", errors="replace")).hexdigest()[:16]
components.append(f"{domain.lstrip('.')}|{name}|{value_hash}")
expiry = _expires_at(cookie)
if expiry is not None and (earliest is None or expiry < earliest):
earliest = expiry
# 浏览器写入 cookie 的顺序并不稳定;排序后哈希才能保证同一份登录态得到同一指纹。
digest = hashlib.sha256()
for component in sorted(components):
digest.update(f"{component}\n".encode("utf-8"))
return CookieSnapshot(
cookie_count=len(cookies),
required_count=required_count,
fingerprint=digest.hexdigest()[:32] if required_count else "",
declared_expires_at=earliest,
)
def snapshot_for_account(account: Any) -> CookieSnapshot | None:
"""Read an account's cookie vault and summarise it, or ``None`` when unreadable.
Shared by the gate and the keepalive manager so both record the exact same
fingerprint for the same vault contents.
"""
cookie_file = getattr(account, "cookie_file", None)
if cookie_file is None:
return None
try:
from gyxx_flow.adapters.browser import BrowserCookieStore
store = BrowserCookieStore(
cookie_file,
getattr(account, "storage_state_file", cookie_file),
)
cookies = store.load_cookies()
except Exception: # noqa: BLE001 - a missing/corrupt vault is itself a signal
return None
return summarize_cookies(
cookies,
required_domains=tuple(getattr(account, "required_cookie_domains", ()) or ()),
required_names=tuple(getattr(account, "required_cookie_names", ()) or ()),
)
# ----------------------------------------------------------------------
# 时间与文本工具
# ----------------------------------------------------------------------
def _parse_datetime(value: Any) -> datetime | None:
if isinstance(value, datetime):
return value if value.tzinfo else value.replace(tzinfo=timezone.utc)
if not isinstance(value, str) or not value.strip():
return None
text = value.strip().replace("Z", "+00:00")
try:
parsed = datetime.fromisoformat(text)
except ValueError:
return None
return parsed if parsed.tzinfo else parsed.replace(tzinfo=timezone.utc)
def _parse_date(value: Any) -> date | None:
if isinstance(value, date) and not isinstance(value, datetime):
return value
if not isinstance(value, str):
return None
try:
return date.fromisoformat(value)
except ValueError:
return None
def _seconds_between(start: datetime | None, end: datetime | None) -> int | None:
if start is None or end is None:
return None
delta = (end - start).total_seconds()
return max(0, int(delta))
def _clip(value: Any, limit: int = _MAX_TEXT_CHARS) -> str:
if value is None:
return ""
text = value if isinstance(value, str) else str(value)
return text[:limit]
def split_step_key(key: str) -> tuple[str, int]:
"""``sync.attempt-2`` -> ``("sync", 2)``; 无法解析时原样当作 step_id。"""
if ".attempt-" in key:
name, _, raw = key.rpartition(".attempt-")
try:
attempt = int(raw)
except ValueError:
return key, 1
if name and attempt > 0:
return name, attempt
return key, 1
# ----------------------------------------------------------------------
# 审计存储
# ----------------------------------------------------------------------
class AuditStore:
"""Idempotent writer for the ``ops_*`` tables.
所有公开写入方法都返回 bool,并且**永不抛出**异常;数据库问题只会导致本次
审计丢失,不会影响工作流执行。
"""
def __init__(
self,
data_root: Path | str,
*,
environment: Mapping[str, str] | None = None,
connect: Any | None = None,
clock: Any | None = None,
) -> None:
self._data_root = Path(data_root).expanduser().resolve()
self._environment = dict(os.environ if environment is None else environment)
self._connect_impl = connect
self._clock = clock or (lambda: datetime.now(timezone.utc))
self._schema_ready = False
self._consecutive_failures = 0
self._cooldown_until = 0.0
self._last_error = ""
self._failure_log = self._data_root / "logs" / "ops-audit" / "audit-failures.log"
# -- 状态 ---------------------------------------------------------
@property
def enabled(self) -> bool:
raw = str(self._environment.get(OPS_AUDIT_ENV, "")).strip().lower()
return raw not in {"0", "off", "false", "no", "disabled"}
@property
def last_error(self) -> str:
return self._last_error
def _host(self) -> str:
override = str(self._environment.get(OPS_AUDIT_HOST_ENV, "")).strip()
if override:
return override
try:
return socket.gethostname()[:128]
except OSError: # pragma: no cover - 极端环境
return ""
def _cooldown_seconds(self) -> int:
raw = str(self._environment.get(OPS_AUDIT_COOLDOWN_ENV, "")).strip()
try:
return max(0, int(raw))
except ValueError:
return _DEFAULT_COOLDOWN_SECONDS
def _in_cooldown(self) -> bool:
import time
return time.monotonic() < self._cooldown_until
# -- 连接 ---------------------------------------------------------
@contextmanager
def _connection(self) -> Iterator[Any]:
if self._connect_impl is not None:
conn = self._connect_impl()
try:
yield conn
finally:
close = getattr(conn, "close", None)
if callable(close):
close()
return
import psycopg # 延迟导入:未安装 psycopg 时整个模块仍可导入
from psycopg.rows import dict_row
dsn, kwargs = database_kwargs(self._environment)
if dsn:
conn = psycopg.connect(dsn, row_factory=dict_row)
else:
conn = psycopg.connect(**kwargs, row_factory=dict_row)
try:
yield conn
finally:
conn.close()
def _ensure_schema(self, conn: Any) -> None:
"""Create the audit objects once, serialising the first DDL pass.
Several workflows can start at the same moment; running CREATE/COMMENT
concurrently lets sessions take DDL locks in different orders and can
deadlock. The advisory lock is only taken when the schema is missing,
so steady-state processes skip the round trip entirely.
"""
if self._schema_ready:
return
with conn.cursor() as cursor:
cursor.execute(
"SELECT 1 FROM pg_class "
"WHERE relname = 'ops_audit_schema' AND relkind = 'r'"
)
initialised = cursor.fetchone() is not None
if not initialised:
with conn.cursor() as cursor:
cursor.execute(
"SELECT pg_advisory_xact_lock(hashtext(%s))",
(_SCHEMA_LOCK_KEY,),
)
cursor.execute(SCHEMA_SQL)
cursor.execute(
"INSERT INTO ops_audit_schema (schema_version) VALUES (%s) "
"ON CONFLICT (schema_version) DO NOTHING",
(SCHEMA_VERSION,),
)
conn.commit()
self._schema_ready = True
# -- 失败处理 -----------------------------------------------------
def _record_failure(self, operation: str, exc: BaseException) -> None:
import time
self._consecutive_failures += 1
self._last_error = f"{type(exc).__name__}: {exc}"[:300]
if self._consecutive_failures >= _MAX_CONSECUTIVE_FAILURES:
self._cooldown_until = time.monotonic() + self._cooldown_seconds()
try:
self._failure_log.parent.mkdir(parents=True, exist_ok=True)
payload = {
"at": self._clock().isoformat(timespec="seconds"),
"operation": operation,
"error": self._last_error,
"consecutive_failures": self._consecutive_failures,
}
with self._failure_log.open("a", encoding="utf-8") as stream:
stream.write(json.dumps(payload, ensure_ascii=False) + "\n")
except OSError:
pass
def _run_guarded(self, operation: str, action: Any) -> bool:
if not self.enabled or self._in_cooldown():
return False
try:
with self._connection() as conn:
self._ensure_schema(conn)
action(conn)
self._consecutive_failures = 0
return True
except Exception as exc: # noqa: BLE001 - 审计绝不影响业务
self._record_failure(operation, exc)
return False
# -- 运行记录 -----------------------------------------------------
def record_run(
self,
record: Mapping[str, Any],
steps: Mapping[str, Any] | None = None,
) -> bool:
"""Upsert one run and its steps from a journal payload or RunRecord dict."""
workflow_id = _clip(record.get("workflow_id"), 200)
run_id = _clip(record.get("run_id"), 200)
business_date = _parse_date(record.get("business_date"))
if not workflow_id or not run_id or business_date is None:
return False
status = _clip(record.get("status"), 32) or "unknown"
mode = _clip(record.get("mode"), 32) or "unknown"
started_at = _parse_datetime(record.get("started_at"))
ended_at = _parse_datetime(record.get("ended_at"))
step_rows = self._step_rows(run_id, steps)
counts = self._step_counts(steps, record)
def action(conn: Any) -> None:
with conn.cursor() as cursor:
cursor.execute(
"""
INSERT INTO ops_workflow_runs (
run_id, workflow_id, business_date, mode, shadow, status,
started_at, ended_at, duration_seconds, error,
step_total, step_success, step_failed, step_skipped,
step_running, journal_path, host, indexed_at
) VALUES (
%(run_id)s, %(workflow_id)s, %(business_date)s, %(mode)s,
%(shadow)s, %(status)s, %(started_at)s, %(ended_at)s,
%(duration_seconds)s, %(error)s, %(step_total)s,
%(step_success)s, %(step_failed)s, %(step_skipped)s,
%(step_running)s, %(journal_path)s, %(host)s, NOW()
)
ON CONFLICT (run_id) DO UPDATE SET
status = EXCLUDED.status,
ended_at = EXCLUDED.ended_at,
duration_seconds = EXCLUDED.duration_seconds,
error = EXCLUDED.error,
step_total = EXCLUDED.step_total,
step_success = EXCLUDED.step_success,
step_failed = EXCLUDED.step_failed,
step_skipped = EXCLUDED.step_skipped,
step_running = EXCLUDED.step_running,
journal_path = EXCLUDED.journal_path,
indexed_at = NOW()
""",
{
"run_id": run_id,
"workflow_id": workflow_id,
"business_date": business_date,
"mode": mode,
"shadow": bool(record.get("shadow", False)),
"status": status,
"started_at": started_at,
"ended_at": ended_at,
"duration_seconds": _seconds_between(started_at, ended_at),
"error": _clip(record.get("error")),
"step_total": counts["total"],
"step_success": counts["success"],
"step_failed": counts["failed"],
"step_skipped": counts["skipped"],
"step_running": counts["running"],
"journal_path": _clip(record.get("journal_path"), 1000),
"host": self._host(),
},
)
if step_rows:
cursor.executemany(
"""
INSERT INTO ops_workflow_run_steps (
run_id, step_id, attempt, status, exit_code,
started_at, ended_at, duration_seconds, error
) VALUES (
%(run_id)s, %(step_id)s, %(attempt)s, %(status)s,
%(exit_code)s, %(started_at)s, %(ended_at)s,
%(duration_seconds)s, %(error)s
)
ON CONFLICT (run_id, step_id, attempt) DO UPDATE SET
status = EXCLUDED.status,
exit_code = EXCLUDED.exit_code,
started_at = COALESCE(
ops_workflow_run_steps.started_at,
EXCLUDED.started_at
),
ended_at = EXCLUDED.ended_at,
duration_seconds = EXCLUDED.duration_seconds,
error = EXCLUDED.error
""",
step_rows,
)
conn.commit()
return self._run_guarded("record_run", action)
@staticmethod
def _step_rows(
run_id: str, steps: Mapping[str, Any] | None
) -> list[dict[str, Any]]:
if not isinstance(steps, Mapping):
return []
rows: list[dict[str, Any]] = []
for key, raw in steps.items():
if not isinstance(raw, Mapping):
continue
step_id, attempt = split_step_key(str(key))
attempt_value = raw.get("attempt")
if isinstance(attempt_value, int) and not isinstance(attempt_value, bool):
attempt = max(1, attempt_value)
started_at = _parse_datetime(raw.get("started_at"))
ended_at = _parse_datetime(raw.get("ended_at"))
exit_code = raw.get("exit_code")
rows.append(
{
"run_id": run_id,
"step_id": _clip(step_id, 200),
"attempt": attempt,
"status": _clip(raw.get("status"), 32),
"exit_code": exit_code if isinstance(exit_code, int) else None,
"started_at": started_at,
"ended_at": ended_at,
"duration_seconds": _seconds_between(started_at, ended_at),
"error": _clip(raw.get("error")),
}
)
return rows
@staticmethod
def _step_counts(
steps: Mapping[str, Any] | None, record: Mapping[str, Any]
) -> dict[str, int]:
counts = {"total": 0, "success": 0, "failed": 0, "skipped": 0, "running": 0}
if isinstance(steps, Mapping) and steps:
for raw in steps.values():
if not isinstance(raw, Mapping):
continue
status = str(raw.get("status", ""))
counts["total"] += 1
if status in counts:
counts[status] += 1
return counts
# 回退到 RunRecord 的 step_counts 投影
projection = record.get("step_counts")
if isinstance(projection, Mapping):
for name in ("success", "failed", "skipped", "running"):
value = projection.get(name)
if isinstance(value, int) and not isinstance(value, bool):
counts[name] = max(0, value)
counts["total"] = counts["success"] + counts["failed"] + counts["skipped"] + counts["running"]
return counts
# -- cookie 事件 ---------------------------------------------------
def record_cookie_event(
self,
*,
account_id: str,
state: str,
snapshot: CookieSnapshot | None = None,
probe_status: str = "",
probe_source: str = "",
detail: str = "",
workflow_id: str = "",
run_id: str = "",
observed_at: datetime | None = None,
) -> bool:
"""Append one observation. 状态必须是白名单值,避免脏数据污染统计。"""
account_id = _clip(account_id, 200)
if not account_id:
return False
state = str(state or "").strip()
if state not in _COOKIE_STATES:
raise ValueError(f"invalid cookie state: {state!r}")
def action(conn: Any) -> None:
with conn.cursor() as cursor:
cursor.execute(
"""
INSERT INTO ops_account_cookie_events (
account_id, observed_at, state, probe_status, probe_source,
cookie_count, required_count, fingerprint, declared_expires_at,
detail, workflow_id, run_id, host
) VALUES (
%(account_id)s, %(observed_at)s, %(state)s, %(probe_status)s,
%(probe_source)s, %(cookie_count)s, %(required_count)s,
%(fingerprint)s, %(declared_expires_at)s, %(detail)s,
%(workflow_id)s, %(run_id)s, %(host)s
)
""",
{
"account_id": account_id,
"observed_at": observed_at or self._clock(),
"state": state,
"probe_status": _clip(probe_status, 64),
"probe_source": _clip(probe_source, 64),
"cookie_count": snapshot.cookie_count if snapshot else None,
"required_count": snapshot.required_count if snapshot else None,
"fingerprint": snapshot.fingerprint if snapshot else "",
"declared_expires_at": (
snapshot.declared_expires_at if snapshot else None
),
"detail": _clip(detail),
"workflow_id": _clip(workflow_id, 200),
"run_id": _clip(run_id, 200),
"host": self._host(),
},
)
conn.commit()
return self._run_guarded("record_cookie_event", action)
# -- 查询(供 CLI) ------------------------------------------------
def query_runs(
self,
*,
workflow_id: str | None = None,
status: str | None = None,
business_date: str | None = None,
limit: int = 50,
) -> list[dict[str, Any]]:
conditions: list[str] = []
params: dict[str, Any] = {"limit": max(1, min(int(limit), 500))}
if workflow_id:
conditions.append("workflow_id = %(workflow_id)s")
params["workflow_id"] = workflow_id
if status:
conditions.append("status = %(status)s")
params["status"] = status
if business_date:
parsed = _parse_date(business_date)
if parsed is not None:
conditions.append("business_date = %(business_date)s")
params["business_date"] = parsed
where = f"WHERE {' AND '.join(conditions)}" if conditions else ""
sql = f"""
SELECT * FROM ops_workflow_runs
{where}
ORDER BY started_at DESC NULLS LAST, run_id DESC
LIMIT %(limit)s
"""
return self._fetch(sql, params)
def query_steps(self, run_id: str) -> list[dict[str, Any]]:
return self._fetch(
"""
SELECT * FROM ops_workflow_run_steps
WHERE run_id = %(run_id)s
ORDER BY step_id, attempt
""",
{"run_id": run_id},
)
def query_cookie_events(
self,
*,
account_id: str | None = None,
state: str | None = None,
limit: int = 50,
) -> list[dict[str, Any]]:
conditions: list[str] = []
params: dict[str, Any] = {"limit": max(1, min(int(limit), 500))}
if account_id:
conditions.append("account_id = %(account_id)s")
params["account_id"] = account_id
if state:
conditions.append("state = %(state)s")
params["state"] = state
where = f"WHERE {' AND '.join(conditions)}" if conditions else ""
sql = f"""
SELECT * FROM ops_account_cookie_events
{where}
ORDER BY observed_at DESC
LIMIT %(limit)s
"""
return self._fetch(sql, params)
def query_cookie_lifetimes(self, *, account_id: str | None = None) -> list[dict[str, Any]]:
if account_id:
return self._fetch(
"""
SELECT * FROM ops_account_cookie_lifetimes
WHERE account_id = %(account_id)s
ORDER BY first_seen_at DESC
""",
{"account_id": account_id},
)
return self._fetch(
"SELECT * FROM ops_account_cookie_lifetimes ORDER BY first_seen_at DESC",
{},
)
def query_cookie_stats(self) -> list[dict[str, Any]]:
return self._fetch(
"SELECT * FROM ops_account_cookie_stats ORDER BY account_id",
{},
)
def _fetch(self, sql: str, params: Mapping[str, Any]) -> list[dict[str, Any]]:
if not self.enabled or self._in_cooldown():
return []
try:
with self._connection() as conn:
self._ensure_schema(conn)
with conn.cursor() as cursor:
cursor.execute(sql, params)
rows = cursor.fetchall()
self._consecutive_failures = 0
return [dict(row) for row in rows]
except Exception as exc: # noqa: BLE001
self._record_failure("query", exc)
return []
# ----------------------------------------------------------------------
# 连接参数(与 PostgresWorkflowConfigStore 同口径)
# ----------------------------------------------------------------------
def database_kwargs(environment: Mapping[str, str]) -> tuple[str | None, dict[str, Any]]:
values = _apply_runtime_policy(environment)
dsn = str(values.get("GYXX_POSTGRES_DSN", "")).strip()
if dsn:
return dsn, {}
required = ("PG_HOST", "PG_PORT", "PG_DB", "PG_USER", "PG_PASSWORD")
missing = [name for name in required if not str(values.get(name, "")).strip()]
if missing:
raise RuntimeError("审计数据库配置缺失: " + ", ".join(missing))
try:
port = int(values["PG_PORT"])
except (TypeError, ValueError) as exc:
raise RuntimeError("PG_PORT 必须是整数") from exc
return None, {
"host": str(values["PG_HOST"]).strip(),
"port": port,
"dbname": str(values["PG_DB"]).strip(),
"user": str(values["PG_USER"]).strip(),
"password": str(values["PG_PASSWORD"]),
}
def _apply_runtime_policy(environment: Mapping[str, str]) -> Mapping[str, str]:
"""复用 RuntimeServicePolicy,保证与控制台/工作流看到同一个数据库地址。"""
try:
from gyxx_flow.adapters import RuntimeServicePolicy
except Exception: # pragma: no cover - 循环导入兜底
return environment
try:
return RuntimeServicePolicy().apply(dict(environment))
except Exception: # pragma: no cover
return environment
def default_store(data_root: Path | str | None = None) -> AuditStore:
"""Best-effort store resolved from the standard data root."""
if data_root is None:
try:
from gyxx_flow.core.config import Settings
data_root = Settings.from_env().data_root
except Exception: # pragma: no cover
data_root = Path.cwd() / "var"
return AuditStore(data_root)
+40
View File
@@ -86,6 +86,7 @@ class RunIndex:
resource = f"ops:run-index:{_digest(record.run_id)}" resource = f"ops:run-index:{_digest(record.run_id)}"
with self._locks.acquire(resource, owner=_owner(), timeout_seconds=30): with self._locks.acquire(resource, owner=_owner(), timeout_seconds=30):
atomic_write_json(destination, record.to_dict()) atomic_write_json(destination, record.to_dict())
_mirror_run_to_audit(self._data_root, record, raw)
return record return record
def get(self, run_id: str) -> RunRecord | None: def get(self, run_id: str) -> RunRecord | None:
@@ -255,3 +256,42 @@ def _digest(value: str) -> str:
import hashlib import hashlib
return hashlib.sha256(value.encode("utf-8")).hexdigest()[:32] return hashlib.sha256(value.encode("utf-8")).hexdigest()[:32]
_AUDIT_STORES: dict[str, Any] = {}
def _audit_store_for(data_root: Path) -> Any:
"""Reuse one store per data root so schema bootstrap and the circuit breaker
survive the short-lived ``RunIndex`` instances created per run."""
key = str(data_root)
store = _AUDIT_STORES.get(key)
if store is None:
from gyxx_flow.ops.audit import AuditStore
store = AuditStore(data_root)
_AUDIT_STORES[key] = store
return store
def _mirror_run_to_audit(
data_root: Path,
record: RunRecord,
journal_payload: Any,
) -> None:
"""Best-effort mirror of a run into the PostgreSQL audit tables.
Auditing is strictly additive: any failure here (missing DSN, unreachable
database, unexpected schema state) is swallowed so that journal indexing and
the workflow itself are never affected.
"""
try:
store = _audit_store_for(data_root)
if not store.enabled:
return
steps = journal_payload.get("steps") if isinstance(journal_payload, dict) else None
store.record_run(record.to_dict(), steps)
except Exception: # noqa: BLE001 - audit mirroring must never raise
return
@@ -0,0 +1,152 @@
-- Migration 001: operational audit tables (workflow runs + login cookie telemetry).
--
-- 本文件由 src/gyxx_flow/ops/audit.py 的 SCHEMA_SQL 导出,请勿手工编辑;
-- 运行时首次写入会自动执行同一份 DDL(幂等),此处仅供 DBA 审阅或预建。
--
-- 用法: psql "$GYXX_POSTGRES_DSN" -f 001_ops_audit.sql
--
-- 说明:
-- ops_workflow_runs 一次工作流运行一行,run.json 的关系型投影
-- ops_workflow_run_steps 每次尝试一行(含重试),保留 exit_code 与错误
-- ops_account_cookie_events cookie 观测事件流,只存指纹不存明文值
-- ops_account_cookie_lifetimes 一处登录态从首次出现到失效(视图)
-- ops_account_cookie_stats 每账号失效规律汇总(视图)
--
-- 日志正文仍留在 var/logs 下,数据库只存结构化元数据与指向路径。
CREATE TABLE IF NOT EXISTS ops_workflow_runs (
run_id TEXT PRIMARY KEY,
workflow_id TEXT NOT NULL,
business_date DATE NOT NULL,
mode TEXT NOT NULL DEFAULT 'unknown',
shadow BOOLEAN NOT NULL DEFAULT FALSE,
status TEXT NOT NULL,
started_at TIMESTAMPTZ,
ended_at TIMESTAMPTZ,
duration_seconds INTEGER,
error TEXT,
step_total INTEGER NOT NULL DEFAULT 0,
step_success INTEGER NOT NULL DEFAULT 0,
step_failed INTEGER NOT NULL DEFAULT 0,
step_skipped INTEGER NOT NULL DEFAULT 0,
step_running INTEGER NOT NULL DEFAULT 0,
journal_path TEXT NOT NULL DEFAULT '',
host TEXT NOT NULL DEFAULT '',
indexed_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS ops_workflow_runs_workflow_idx
ON ops_workflow_runs (workflow_id, started_at DESC);
CREATE INDEX IF NOT EXISTS ops_workflow_runs_status_idx
ON ops_workflow_runs (status, started_at DESC);
CREATE INDEX IF NOT EXISTS ops_workflow_runs_business_date_idx
ON ops_workflow_runs (business_date, workflow_id);
CREATE TABLE IF NOT EXISTS ops_workflow_run_steps (
run_id TEXT NOT NULL
REFERENCES ops_workflow_runs (run_id) ON DELETE CASCADE,
step_id TEXT NOT NULL,
attempt INTEGER NOT NULL DEFAULT 1,
status TEXT NOT NULL DEFAULT '',
exit_code INTEGER,
started_at TIMESTAMPTZ,
ended_at TIMESTAMPTZ,
duration_seconds INTEGER,
error TEXT,
PRIMARY KEY (run_id, step_id, attempt)
);
CREATE INDEX IF NOT EXISTS ops_workflow_run_steps_step_idx
ON ops_workflow_run_steps (step_id, status);
CREATE TABLE IF NOT EXISTS ops_account_cookie_events (
id BIGSERIAL PRIMARY KEY,
account_id TEXT NOT NULL,
observed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
state TEXT NOT NULL,
probe_status TEXT NOT NULL DEFAULT '',
probe_source TEXT NOT NULL DEFAULT '',
cookie_count INTEGER,
required_count INTEGER,
fingerprint TEXT NOT NULL DEFAULT '',
declared_expires_at TIMESTAMPTZ,
detail TEXT NOT NULL DEFAULT '',
workflow_id TEXT NOT NULL DEFAULT '',
run_id TEXT NOT NULL DEFAULT '',
host TEXT NOT NULL DEFAULT ''
);
CREATE INDEX IF NOT EXISTS ops_account_cookie_events_account_idx
ON ops_account_cookie_events (account_id, observed_at DESC);
CREATE INDEX IF NOT EXISTS ops_account_cookie_events_state_idx
ON ops_account_cookie_events (state, observed_at DESC);
CREATE INDEX IF NOT EXISTS ops_account_cookie_events_fingerprint_idx
ON ops_account_cookie_events (account_id, fingerprint);
-- 一段「cookie 代」的生命周期:同一 fingerprint 从首次出现到失效。
-- first_failed_at 只取最后一次 healthy 之后的失败,避免恢复后再次失败时口径前移。
CREATE OR REPLACE VIEW ops_account_cookie_lifetimes AS
WITH per_fingerprint AS (
SELECT
account_id,
fingerprint,
MIN(observed_at) AS first_seen_at,
MIN(observed_at) FILTER (WHERE state = 'healthy') AS first_healthy_at,
MAX(observed_at) FILTER (WHERE state = 'healthy') AS last_healthy_at,
MAX(declared_expires_at) AS declared_expires_at
FROM ops_account_cookie_events
WHERE fingerprint <> ''
GROUP BY account_id, fingerprint
)
SELECT
p.account_id,
p.fingerprint,
p.first_seen_at,
p.first_healthy_at,
p.last_healthy_at,
p.declared_expires_at,
f.first_failed_at,
(f.first_failed_at IS NOT NULL) AS expired,
CASE
WHEN p.first_healthy_at IS NOT NULL AND p.last_healthy_at IS NOT NULL
THEN EXTRACT(EPOCH FROM (p.last_healthy_at - p.first_healthy_at))::BIGINT
END AS healthy_span_seconds,
CASE
WHEN p.first_healthy_at IS NOT NULL AND f.first_failed_at IS NOT NULL
THEN EXTRACT(EPOCH FROM (f.first_failed_at - p.first_healthy_at))::BIGINT
END AS lifetime_seconds,
CASE
WHEN p.last_healthy_at IS NOT NULL AND f.first_failed_at IS NOT NULL
THEN EXTRACT(EPOCH FROM (f.first_failed_at - p.last_healthy_at))::BIGINT
END AS healthy_to_failure_seconds
FROM per_fingerprint p
LEFT JOIN LATERAL (
SELECT MIN(e.observed_at) AS first_failed_at
FROM ops_account_cookie_events e
WHERE e.account_id = p.account_id
AND e.fingerprint = p.fingerprint
AND e.state IN ('expired', 'not_logged_in')
AND (p.last_healthy_at IS NULL OR e.observed_at > p.last_healthy_at)
) f ON TRUE;
-- 按账号汇总的失效规律:平均/中位/最短/最长寿命,直接支撑「多久失效一次」。
CREATE OR REPLACE VIEW ops_account_cookie_stats AS
SELECT
account_id,
COUNT(*) AS sessions,
COUNT(*) FILTER (WHERE expired) AS expired_sessions,
AVG(lifetime_seconds)::BIGINT AS avg_lifetime_seconds,
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY lifetime_seconds)::BIGINT
AS median_lifetime_seconds,
MIN(lifetime_seconds) AS min_lifetime_seconds,
MAX(lifetime_seconds) AS max_lifetime_seconds,
MAX(first_failed_at) AS last_failed_at,
MAX(last_healthy_at) AS last_healthy_at
FROM ops_account_cookie_lifetimes
WHERE lifetime_seconds IS NOT NULL
GROUP BY account_id;
CREATE TABLE IF NOT EXISTS ops_audit_schema (
schema_version INTEGER PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
@@ -1,3 +1,5 @@
import json
import time
from datetime import datetime from datetime import datetime
from pathlib import Path from pathlib import Path
from types import SimpleNamespace from types import SimpleNamespace
@@ -33,7 +35,7 @@ def test_refresh_then_export_waits_once_after_all_accounts(monkeypatch):
monkeypatch.setattr(chanmama, "export_video_data", lambda _driver, mtime: events.append(("export", mtime)) or "x.xlsx") monkeypatch.setattr(chanmama, "export_video_data", lambda _driver, mtime: events.append(("export", mtime)) or "x.xlsx")
monkeypatch.setattr(chanmama, "parse_chanmama_excel", lambda _path: ([{"title": "video"}], None)) monkeypatch.setattr(chanmama, "parse_chanmama_excel", lambda _path: ([{"title": "video"}], None))
records, summaries = chanmama.refresh_then_export_accounts(object(), urls, wait_seconds=600) records, summaries, _driver = chanmama.refresh_then_export_accounts(object(), urls, wait_seconds=600)
assert len(records) == 2 assert len(records) == 2
assert [item["clicked"] for item in summaries] == [2, 0] assert [item["clicked"] for item in summaries] == [2, 0]
@@ -54,7 +56,7 @@ def test_refresh_then_export_skips_wait_when_nothing_needs_refresh(monkeypatch):
monkeypatch.setattr(chanmama, "_max_excel_mtime", lambda: 0.0) monkeypatch.setattr(chanmama, "_max_excel_mtime", lambda: 0.0)
monkeypatch.setattr(chanmama, "export_video_data", lambda *_args: None) monkeypatch.setattr(chanmama, "export_video_data", lambda *_args: None)
records, _ = chanmama.refresh_then_export_accounts(object(), ["account-1"], wait_seconds=600) records, _, _driver = chanmama.refresh_then_export_accounts(object(), ["account-1"], wait_seconds=600)
assert records == [] assert records == []
assert sleeps == [] assert sleeps == []
@@ -159,7 +161,6 @@ def test_missing_get_data_button_does_not_trigger_refresh_wait(monkeypatch):
chanmama.refresh_then_export_accounts(object(), ["account-1"], wait_seconds=600) chanmama.refresh_then_export_accounts(object(), ["account-1"], wait_seconds=600)
assert sleeps == [] assert sleeps == []
def test_create_driver_uses_runtime_cdp_profile_and_download_bindings(monkeypatch, tmp_path): def test_create_driver_uses_runtime_cdp_profile_and_download_bindings(monkeypatch, tmp_path):
calls = [] calls = []
expected = object() expected = object()
@@ -183,7 +184,9 @@ def test_create_driver_uses_runtime_cdp_profile_and_download_bindings(monkeypatc
"profile_dir": str(tmp_path / "profile"), "profile_dir": str(tmp_path / "profile"),
"download_dir": str(tmp_path / "downloads"), "download_dir": str(tmp_path / "downloads"),
"cdp_url": "http://127.0.0.1:22002", "cdp_url": "http://127.0.0.1:22002",
"stealthy": True, "stealthy": False,
"real_chrome": True,
"extra_flags": ["--no-proxy-server"],
}] }]
@@ -206,8 +209,10 @@ def test_create_driver_starts_scrapling_without_cdp(monkeypatch, tmp_path):
assert chanmama.create_driver() is expected assert chanmama.create_driver() is expected
assert calls[0]["cdp_url"] is None assert calls[0]["cdp_url"] is None
assert calls[0]["stealthy"] is True # 蝉妈妈 WAF 拉黑代理出口且压制 stealthy 补丁,必须真 Chrome + 直连。
assert calls[0]["real_chrome"] is False assert calls[0]["stealthy"] is False
assert calls[0]["real_chrome"] is True
assert calls[0]["extra_flags"] == ["--no-proxy-server"]
def test_create_driver_falls_back_when_bound_cdp_is_not_listening( def test_create_driver_falls_back_when_bound_cdp_is_not_listening(
@@ -241,8 +246,8 @@ def test_create_driver_falls_back_when_bound_cdp_is_not_listening(
"http://127.0.0.1:22002", "http://127.0.0.1:22002",
None, None,
] ]
assert all(call["stealthy"] is True for call in calls) assert all(call["stealthy"] is False for call in calls)
assert calls[1]["real_chrome"] is False assert all(call["real_chrome"] is True for call in calls)
def test_create_driver_keeps_explicit_remote_cdp_fail_closed( def test_create_driver_keeps_explicit_remote_cdp_fail_closed(
@@ -761,7 +766,7 @@ def test_refresh_then_export_relogin_on_expired_session(monkeypatch):
chanmama, "parse_chanmama_excel", lambda _path: ([{"title": "v"}], None) chanmama, "parse_chanmama_excel", lambda _path: ([{"title": "v"}], None)
) )
records, summaries = chanmama.refresh_then_export_accounts( records, summaries, _driver = chanmama.refresh_then_export_accounts(
object(), ["account-1", "account-2"], wait_seconds=0 object(), ["account-1", "account-2"], wait_seconds=0
) )
@@ -791,7 +796,7 @@ def test_refresh_then_export_no_relogin_for_other_errors(monkeypatch):
chanmama, "export_video_data", lambda _driver, mtime: None chanmama, "export_video_data", lambda _driver, mtime: None
) )
records, _summaries = chanmama.refresh_then_export_accounts( records, _summaries, _driver = chanmama.refresh_then_export_accounts(
object(), ["account-1"], wait_seconds=0 object(), ["account-1"], wait_seconds=0
) )
@@ -889,3 +894,192 @@ def test_wait_video_list_ready_tolerates_transient_script_errors(monkeypatch):
return True return True
chanmama._wait_video_list_ready(FlakyDriver(), 2) # 不抛即通过 chanmama._wait_video_list_ready(FlakyDriver(), 2) # 不抛即通过
def test_wait_for_exported_excel_raises_when_browser_closed_without_file(monkeypatch):
"""浏览器被外部关闭且文件未落盘: 立刻抛出关闭异常,不再空等满超时。"""
class _DeadDriver:
def execute_script(self, _script, *_args):
raise RuntimeError(
"Page.goto: Target page, context or browser has been closed"
)
monkeypatch.setattr(chanmama.time, "time", lambda: 1000.0)
monkeypatch.setattr(
chanmama.time,
"sleep",
lambda _seconds: pytest.fail("浏览器已死不应继续轮询等待"),
)
monkeypatch.setattr(chanmama, "_find_new_excel_after", lambda _pre_mtime: None)
with pytest.raises(RuntimeError, match="closed"):
chanmama._wait_for_exported_excel(0.0, driver=_DeadDriver())
def test_wait_for_exported_excel_returns_file_if_landed_before_close(monkeypatch):
"""关闭前下载已完整落盘: 仍返回文件路径,不因浏览器死亡误报失败。"""
class _DeadDriver:
def execute_script(self, _script, *_args):
raise RuntimeError(
"Download.save_as: Target page, context or browser has been closed"
)
landed = None
def fake_find(_pre_mtime):
return landed
def fake_complete(path):
return True
monkeypatch.setattr(chanmama.time, "time", lambda: 1000.0)
monkeypatch.setattr(chanmama, "_find_new_excel_after", fake_find)
monkeypatch.setattr(chanmama, "_download_complete", fake_complete)
landed = "/downloads/关闭前已落盘.xlsx"
assert chanmama._wait_for_exported_excel(0.0, driver=_DeadDriver()) == landed
def test_refresh_then_export_rebuilds_browser_when_closed_mid_run(monkeypatch):
"""导出中途浏览器被外部关闭: 重建驱动+恢复会话后重试当前博主一次。"""
events = []
navigate_calls = []
dead_driver = object()
live_driver = object()
def fake_navigate(current, url):
navigate_calls.append((current, url))
if current is dead_driver and len(navigate_calls) == 3:
# 刷新阶段两个博主都成功后,导出阶段第一次导航即发现浏览器死亡
raise RuntimeError(
"蝉妈妈目标页加载失败: x.html; Page.goto: "
"Target page, context or browser has been closed"
)
monkeypatch.setattr(chanmama, "navigate_to_target", fake_navigate)
monkeypatch.setattr(
chanmama,
"refresh_zero_exposure_videos",
lambda _driver: {
"clicked": 0, "pending": 0, "zero_without_button": 0,
"unconfirmed": 0, "pages": 1,
},
)
monkeypatch.setattr(chanmama.time, "sleep", lambda _seconds: None)
monkeypatch.setattr(chanmama, "_max_excel_mtime", lambda: 0.0)
monkeypatch.setattr(
chanmama, "export_video_data", lambda _driver, mtime: "x.xlsx"
)
monkeypatch.setattr(
chanmama, "parse_chanmama_excel", lambda _path: ([{"title": "v"}], None)
)
monkeypatch.setattr(chanmama, "verify_login", lambda _driver: True)
monkeypatch.setattr(
chanmama, "create_driver", lambda: events.append("rebuild") or live_driver
)
monkeypatch.setattr(
chanmama, "login",
lambda _driver: events.append("login") or True,
)
monkeypatch.setattr(
chanmama, "load_cookies",
lambda _driver: (_ for _ in ()).throw(
AssertionError("profile 直接恢复,不应走 cookie 文件")
),
)
class _DeadDriver:
def quit(self):
events.append("quit-dead")
dead_driver = _DeadDriver()
records, _summaries, final_driver = chanmama.refresh_then_export_accounts(
dead_driver, ["account-1", "account-2"], wait_seconds=0
)
assert events == ["quit-dead", "rebuild"] # 旧驱动清理+重建一次
assert final_driver is live_driver # 返回重建后的驱动供收尾关闭
assert len(records) == 2 # 两个博主最终都导出成功
# 第 3 次导航(导出博主 1)发现死亡,其后所有导航都在重建后的驱动上
assert all(current is live_driver for current, _ in navigate_calls[3:])
def test_refresh_then_export_rebuilds_only_once(monkeypatch):
"""环境持续杀浏览器时只重建一次,后续博主直接失败不再死循环。"""
rebuilds = []
def always_closed_navigate(_driver, _url):
raise RuntimeError(
"Page.goto: Target page, context or browser has been closed"
)
class _Driver:
def quit(self):
pass
monkeypatch.setattr(chanmama, "navigate_to_target", always_closed_navigate)
monkeypatch.setattr(
chanmama, "create_driver", lambda: rebuilds.append(1) or _Driver()
)
monkeypatch.setattr(chanmama, "verify_login", lambda _driver: True)
monkeypatch.setattr(chanmama.time, "sleep", lambda _seconds: None)
records, _summaries, _driver = chanmama.refresh_then_export_accounts(
_Driver(), ["account-1", "account-2"], wait_seconds=0
)
assert len(rebuilds) == 1
assert records == []
def test_save_cookies_writes_adapter_list_format(monkeypatch, tmp_path):
"""cookie 文件与 vault/适配器共用,必须写裸列表而不是字典。"""
cookie_file = tmp_path / "cookies.json"
monkeypatch.setattr(chanmama, "COOKIE_FILE", str(cookie_file))
class _Driver:
@staticmethod
def get_cookies():
return [{"name": "session", "domain": ".chanmama.com"}]
assert chanmama.save_cookies(_Driver()) is True
payload = json.loads(cookie_file.read_text(encoding="utf-8"))
assert payload == [{"name": "session", "domain": ".chanmama.com"}]
def test_load_cookies_accepts_list_and_legacy_dict(monkeypatch, tmp_path):
cookie_file = tmp_path / "cookies.json"
injected = []
class _Driver:
@staticmethod
def get(_url):
return None
@staticmethod
def add_cookie(cookie):
injected.append(cookie)
monkeypatch.setattr(chanmama, "COOKIE_FILE", str(cookie_file))
monkeypatch.setattr(chanmama.time, "sleep", lambda _seconds: None)
cookie_file.write_text(
json.dumps([{"name": "a", "domain": ".chanmama.com"}]),
encoding="utf-8",
)
assert chanmama.load_cookies(_Driver()) is True
assert [c["name"] for c in injected] == ["a"]
injected.clear()
cookie_file.write_text(
json.dumps(
{"cookies": [{"name": "b", "domain": ".chanmama.com"}], "timestamp": time.time()}
),
encoding="utf-8",
)
assert chanmama.load_cookies(_Driver()) is True
assert [c["name"] for c in injected] == ["b"]
@@ -125,6 +125,57 @@ def test_style_records_sum_product_rows_and_keep_zero_styles() -> None:
assert records[1]["matched_product_rows"] == 0 assert records[1]["matched_product_rows"] == 0
def test_style_extraction_maps_hidden_columns_via_visible_positions() -> None:
class Frame:
def __init__(self) -> None:
self.script = ""
self.args = {}
def evaluate(self, script, args):
self.script = script
self.args = args
return {"ok": True, "rows": []}
frame = Frame()
backfill.extract_style_rows(frame, requested_codes=("10416",))
assert "window._Cols" in frame.script
assert "visibleIdx.indexOf" in frame.script
assert "cells[stylePos]" in frame.script
assert "cells.length <= Math.max(stylePos, salesPos, returnsPos)" in frame.script
def test_ensure_custom_columns_visible_only_touches_allsku_frames() -> None:
class AllskuFrame:
url = "https://bi.erp321.com/app/report/subject/itemskuanalysis/allsku.aspx?r=1"
def __init__(self) -> None:
self.scripts: list[str] = []
def evaluate(self, script):
self.scripts.append(script)
return 2
class OtherFrame:
url = "https://bi.erp321.com/app/report/subject/itemskuanalysis/filter.aspx"
def evaluate(self, script):
raise AssertionError("must not touch non-allsku frames")
class Page:
frames = [OtherFrame(), AllskuFrame()]
page = Page()
changed = backfill.ensure_custom_columns_visible(page)
assert changed == 2
allsku = page.frames[1]
assert len(allsku.scripts) == 1
assert "'自定义销量'" in allsku.scripts[0]
assert "'自定义退货量'" in allsku.scripts[0]
assert "_RptRefresh" in allsku.scripts[0]
def test_range_collector_queries_once_per_date_not_once_per_style(monkeypatch) -> None: def test_range_collector_queries_once_per_date_not_once_per_style(monkeypatch) -> None:
filter_frame = object() filter_frame = object()
monkeypatch.setattr( monkeypatch.setattr(
@@ -0,0 +1,77 @@
import unittest
from unittest import mock
import collect_erp_yesterday_metrics as erp_mod
class _StubFrame:
"""frame.evaluate 的最小桩: 记录入参并返回固定 payload。"""
def __init__(self, payload):
self._payload = payload
self.calls = []
def evaluate(self, script, *args):
self.calls.append((script, args))
return self._payload
class ErpDailyHeaderRoutingTest(unittest.TestCase):
"""2026-10-11 聚水潭模板重置隐藏 38 列后, 固定可见索引 45/57 错位到
「今日实退货量/今日仅退款量」, daily 销量全 0。daily 必须走表头名匹配,
且不能落入 backfill 的 _extract_by_header (自定义列 SPU 求和) 分支。"""
def test_daily_mode_uses_yesterday_headers(self):
with mock.patch.object(erp_mod, "MODE", "daily"):
self.assertEqual(
erp_mod._sales_returns_idx(),
("昨日销量", "昨日退货量"),
)
def test_weekly_mode_keeps_legacy_index(self):
with mock.patch.object(erp_mod, "MODE", "weekly"):
self.assertEqual(
erp_mod._sales_returns_idx(),
("__idx__45", "__idx__57"),
)
def test_backfill_mode_keeps_custom_headers(self):
with mock.patch.object(erp_mod, "MODE", "backfill"):
self.assertEqual(
erp_mod._sales_returns_idx(),
("自定义销量", "自定义退货量"),
)
def test_daily_header_keys_bypass_extract_by_header(self):
payload = {
"ok": True,
"yesterday_sales": 2,
"yesterday_returns": 0,
"bodyText": "",
}
frame = _StubFrame(payload)
with mock.patch.object(erp_mod, "MODE", "daily"):
metrics, _body = erp_mod._extract_with_text(frame)
self.assertEqual(metrics["yesterday_sales"], 2)
self.assertEqual(len(frame.calls), 1)
script, args = frame.calls[0]
js_args = args[0] if args else script
self.assertEqual(js_args["mode"], "header")
self.assertEqual(js_args["salesHeader"], "昨日销量")
self.assertEqual(js_args["returnsHeader"], "昨日退货量")
def test_backfill_mode_routes_to_extract_by_header(self):
frame = _StubFrame({})
with mock.patch.object(erp_mod, "MODE", "backfill"):
with mock.patch.object(
erp_mod,
"_extract_by_header",
return_value={"ok": True, "yesterday_sales": 5, "summary_text": "X"},
) as by_header:
metrics, _body = erp_mod._extract_with_text(frame)
by_header.assert_called_once()
self.assertEqual(metrics["yesterday_sales"], 5)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,191 @@
# -*- coding: utf-8 -*-
"""聚水潭销售主题分析(按日)采集脚本的纯解析与 CLI 参数测试。"""
from __future__ import annotations
import sys
from pathlib import Path
import pytest
from gyxx_flow.modules import product_commerce
from gyxx_flow.modules.product_commerce import (
collect_erp_sales_theme_daily as collector,
)
from gyxx_flow.modules.product_commerce import db as product_db
HEADER = [
"日期", "销售单数", "销售数量", "实发数量", "实发金额", "销售金额",
"退货单数", "退货数量", "实退数量", "退货金额", "实退金额",
"净销售单数", "净销量", "净销售额", "运费收入", "运费支出", "实发单数",
]
def _write_workbook(path: Path, rows: list[list]) -> None:
import openpyxl
workbook = openpyxl.Workbook()
sheet = workbook.active
for row in rows:
sheet.append(row)
workbook.save(str(path))
workbook.close()
def test_parse_sales_theme_workbook_roundtrip(tmp_path: Path) -> None:
xlsx = tmp_path / "销售主题分析_按日_1.xlsx"
_write_workbook(
xlsx,
[
HEADER,
["2026-10-11", "649", 655, 646, "173,129.23", 175752.83,
15, 15, 0, 2677.47, "0.00", 634, 640, 173075.36, "20.00", None, 640],
["2026-10-10", 690, 691, 685, 166761.58, 168185.97,
9, 9, 0, 2656.71, 0, 681, 686, 166529.13, None, 1.6, 685],
["##", 1339, 1346, 1331, 339890.81, 343938.8,
24, 24, 0, 5334.18, 0, 1315, 1326, 339604.49, 20, 1.6, 1325],
],
)
records = collector.parse_sales_theme_workbook(xlsx)
assert [record["stat_date"] for record in records] == ["2026-10-11", "2026-10-10"]
first = records[0]
assert first["sales_order_count"] == 649
assert first["sales_qty"] == 655
assert first["shipped_amount"] == 173129.23
assert first["sales_amount"] == 175752.83
assert first["return_amount"] == 2677.47
assert first["returned_amount"] == 0.0
assert first["net_sales_order_count"] == 634
assert first["net_sales_amount"] == 173075.36
assert first["freight_income"] == 20.0
assert first["freight_expense"] is None
assert first["shipped_order_count"] == 640
# 合计行(##)必须被跳过
assert len(records) == 2
def test_parse_sales_theme_workbook_rejects_missing_date_column(tmp_path: Path) -> None:
xlsx = tmp_path / "bad.xlsx"
_write_workbook(xlsx, [["销售单数", "销售数量"], [1, 2]])
with pytest.raises(ValueError):
collector.parse_sales_theme_workbook(xlsx)
def test_parse_number_and_amount_helpers() -> None:
assert collector.parse_number("1,339") == 1339
assert collector.parse_number(None) == 0
assert collector.parse_number("") == 0
assert collector.parse_amount(None) is None
assert collector.parse_amount("") is None
assert collector.parse_amount("1.6") == 1.6
assert collector.parse_amount(2677.467) == 2677.47
class _CaptureCursor:
def __init__(self) -> None:
self.batches: list[tuple] = []
def __enter__(self):
return self
def __exit__(self, *exc_info):
return False
class _CaptureConnection:
def __init__(self) -> None:
self.cursor_obj = _CaptureCursor()
def cursor(self):
return self.cursor_obj
def test_upsert_sales_theme_dedups_and_normalizes(monkeypatch: pytest.MonkeyPatch) -> None:
captured: dict[str, object] = {}
def fake_execute_values(cur, sql, rows, template=None):
captured["sql"] = sql
captured["rows"] = list(rows)
monkeypatch.setattr(product_db, "execute_values", fake_execute_values)
records = [
{
"stat_date": "2026-10-11",
"sales_order_count": 649,
"freight_income": "20.00",
},
{
# 同日重复: 整体覆盖前一条(同 upsert_erp_all_shop 语义), 数值缺省为 0
"stat_date": "2026-10-11",
"sales_order_count": 999,
"brands": ["光影", "光影行星", "光影"],
"shops": ["天猫-光影行星旗舰店", "JD光影行星旗舰店"],
"source_file": "var/raw/erp销售主题分析日报/x.xlsx",
},
{
"stat_date": "2026-10-10",
"sales_qty": "691",
},
]
conn = _CaptureConnection()
written = product_db.upsert_erp_sales_theme_daily_metrics(conn, records)
assert written == 2
rows = captured["rows"]
assert len(rows) == 2
by_date = {row[0]: row for row in rows}
# 同日重复记录取最后一条, 数字字段缺省为 0, 金额字符串转 float
assert by_date["2026-10-11"][1] == 999
assert by_date["2026-10-10"][2] == 691
tenth = by_date["2026-10-10"]
# NOT NULL 金额列缺省(导出空单元格)落 0; 运费两列允许 NULL
assert tenth[9] == 0 # return_amount
assert tenth[15] is None # freight_income
assert tenth[16] is None # freight_expense
eleventh = by_date["2026-10-11"]
assert eleventh[-4] == "发货时间" # date_basis 默认
assert eleventh[-3] == ["光影", "光影行星"] # brands 去重排序
assert eleventh[-2] == ["JD光影行星旗舰店", "天猫-光影行星旗舰店"]
assert eleventh[-1] == "var/raw/erp销售主题分析日报/x.xlsx"
# upsert 目标表与主键在 SQL 里声明
sql = str(captured["sql"])
assert "erp_sales_theme_daily_metrics" in sql
assert "ON CONFLICT (stat_date)" in sql
def test_business_scope_constants_are_consistent() -> None:
assert len(collector.BRANDS) == 3
assert sorted(collector.BRANDS) == ["gyxx", "光影", "光影行星"]
assert len(collector.SHOP_NAMES) == len(set(collector.SHOP_NAMES)) > 0
for name in collector.SHOP_NAMES:
assert name == name.strip()
def test_resolve_window_variants() -> None:
args = collector.parse_args(["--date", "2026-10-11"])
start, end = collector.resolve_window(args)
assert (start.isoformat(), end.isoformat()) == ("2026-10-11", "2026-10-11")
args = collector.parse_args(["--date", "2026-10-11", "--days", "7"])
start, end = collector.resolve_window(args)
assert (start.isoformat(), end.isoformat()) == ("2026-10-05", "2026-10-11")
args = collector.parse_args(["--from", "2025-06-01", "--to", "2026-01-01"])
start, end = collector.resolve_window(args)
assert (start.isoformat(), end.isoformat()) == ("2025-06-01", "2026-01-01")
with pytest.raises(SystemExit):
collector.resolve_window(collector.parse_args(["--from", "2026-01-01"]))
with pytest.raises(SystemExit):
collector.resolve_window(
collector.parse_args(["--from", "2026-01-01", "--to", "2025-01-01"])
)
def test_module_uses_shared_cookie_skip_exit_code() -> None:
from gyxx_flow.core.exit_codes import COOKIE_SKIP_EXIT_CODE
assert COOKIE_SKIP_EXIT_CODE == 75
assert sys.modules[product_commerce.__name__] is not None
@@ -237,6 +237,62 @@ def test_login_if_needed_fills_the_password_form_inside_an_iframe(monkeypatch) -
assert frame.submit.click_count == 1 assert frame.submit.click_count == 1
def test_login_if_needed_accepts_agreement_popup_and_resubmits(monkeypatch) -> None:
events: list[str] = []
class Locator:
def __init__(self, name: str) -> None:
self.name = name
def fill(self, _value: str) -> None:
events.append(f"fill:{self.name}")
def click(self, timeout: int = 0) -> None:
events.append(f"click:{self.name}")
username = Locator("username")
password = Locator("password")
submit = Locator("submit")
page = SimpleNamespace(wait_for_timeout=lambda ms: events.append(f"wait:{ms}"))
monkeypatch.setattr(baibu, "_authenticated", lambda *_args: False)
monkeypatch.setattr(baibu, "_is_login_url", lambda _url: True)
def fake_form(_page, *, use_password_form):
return (_page, username, password, submit)
monkeypatch.setattr(baibu, "_find_password_login_form", fake_form)
monkeypatch.setattr(
baibu,
"_credentials_from_environment",
lambda: (baibu._LoginCredentials(username="acct", secret_value="secret"), ()),
)
agreement = {"visible": True}
monkeypatch.setattr(
baibu, "_is_login_agreement_visible", lambda _page: agreement["visible"]
)
monkeypatch.setattr(
baibu,
"_click_login_agreement_accept",
lambda _page: agreement.update(visible=False) or True,
)
assert baibu._login_if_needed(_entry(), page) is True
# First submit is swallowed by the 服务协议 popup; after accepting it the
# credentials are re-filled and submitted a second time.
assert events == [
"fill:username",
"fill:password",
"click:submit",
"wait:2000",
"fill:username",
"fill:password",
"click:submit",
"wait:2000",
]
def test_manual_login_waits_without_filling_the_password_form(monkeypatch) -> None: def test_manual_login_waits_without_filling_the_password_form(monkeypatch) -> None:
monkeypatch.setattr(baibu, "_authenticated", lambda _entry, _page: False) monkeypatch.setattr(baibu, "_authenticated", lambda _entry, _page: False)
frame = _LoginFrame() frame = _LoginFrame()
@@ -301,6 +357,14 @@ def test_login_second_factor_marker_is_detected(monkeypatch) -> None:
assert baibu._login_second_factor_marker(SimpleNamespace()) == "短信验证" assert baibu._login_second_factor_marker(SimpleNamespace()) == "短信验证"
def test_login_slider_captcha_marker_is_detected(monkeypatch) -> None:
monkeypatch.setattr(
baibu, "_page_text", lambda _page: "向右拖动滑块填充拼图完成验证"
)
assert baibu._login_second_factor_marker(SimpleNamespace()) == "滑块"
def test_persist_account_state_filters_cookies_to_the_account_domains( def test_persist_account_state_filters_cookies_to_the_account_domains(
monkeypatch, monkeypatch,
tmp_path: Path, tmp_path: Path,
@@ -351,6 +415,289 @@ def test_persist_account_state_filters_cookies_to_the_account_domains(
] ]
def _browser_spy(calls: list[str]):
class Page:
def goto(self, *_args, **_kwargs) -> None:
calls.append("goto")
class Context:
pages = [Page()]
def new_page(self):
return Page()
class Browser:
context = Context()
def start(self) -> None:
calls.append("start")
def persist_bound_state(self) -> None:
calls.append("persist-bound")
def close(self) -> None:
calls.append("close")
return Browser()
def test_apply_entry_continues_when_account_vault_refresh_fails(monkeypatch) -> None:
calls: list[str] = []
monkeypatch.setattr(baibu, "_start_browser", lambda: _browser_spy(calls))
monkeypatch.setattr(baibu, "_login_if_needed", lambda *_args, **_kwargs: False)
monkeypatch.setattr(baibu, "_wait_for_authenticated", lambda *_args, **_kwargs: None)
def fail_vault_refresh(_browser, _entry, **_kwargs):
calls.append("persist-fail")
raise baibu.BaibuApplyError(
"测试入口 登录后未获得可用的账号 cookie(域内 cookie:无;"
"要求含:unb、cookie2、_tb_token_),未更新账号登录态"
)
monkeypatch.setattr(baibu, "_persist_account_state", fail_vault_refresh)
def record_explicit_login(_entry, _page, *, manual_login):
calls.append("explicit-login")
monkeypatch.setattr(baibu, "_explicit_password_login", record_explicit_login)
def record_open(_page, _entry):
calls.append("open")
return None
monkeypatch.setattr(baibu, "_open_batch_import", record_open)
monkeypatch.setattr(baibu, "_upload_template", lambda *_args: calls.append("upload"))
monkeypatch.setattr(baibu, "_wait_import_result", lambda *_args, **_kwargs: calls.append("result"))
monkeypatch.setattr(baibu, "_complete_draft_products", lambda *_args: calls.append("drafts"))
monkeypatch.setattr(baibu.time, "sleep", lambda _seconds: None)
baibu._apply_entry(_entry(), Path("template.xlsx"))
assert calls == [
"start",
"goto",
"persist-bound",
"persist-fail",
"explicit-login",
"persist-fail",
"open",
"upload",
"result",
"drafts",
"close",
]
def test_apply_entry_retries_through_explicit_login_when_page_never_shows_login(
monkeypatch,
) -> None:
calls: list[str] = []
monkeypatch.setattr(baibu, "_start_browser", lambda: _browser_spy(calls))
monkeypatch.setattr(baibu, "_login_if_needed", lambda *_args, **_kwargs: False)
attempts = {"count": 0}
def fail_first_authentication(*_args, **_kwargs):
attempts["count"] += 1
if attempts["count"] == 1:
raise baibu.BaibuApplyError("测试入口 当前未登录对应店铺,已停止")
calls.append("authenticated")
monkeypatch.setattr(baibu, "_wait_for_authenticated", fail_first_authentication)
def record_explicit_login(_entry, _page, *, manual_login):
calls.append("explicit-login")
monkeypatch.setattr(baibu, "_explicit_password_login", record_explicit_login)
monkeypatch.setattr(
baibu,
"_persist_account_state",
lambda *_args, **_kwargs: calls.append("persist-ok") or True,
)
def record_open(_page, _entry):
calls.append("open")
return None
monkeypatch.setattr(baibu, "_open_batch_import", record_open)
monkeypatch.setattr(baibu, "_upload_template", lambda *_args: calls.append("upload"))
monkeypatch.setattr(baibu, "_wait_import_result", lambda *_args, **_kwargs: calls.append("result"))
monkeypatch.setattr(baibu, "_complete_draft_products", lambda *_args: calls.append("drafts"))
monkeypatch.setattr(baibu.time, "sleep", lambda _seconds: None)
baibu._apply_entry(_entry(), Path("template.xlsx"))
assert attempts["count"] == 1
assert calls == [
"start",
"goto",
"explicit-login",
"persist-bound",
"persist-ok",
"open",
"upload",
"result",
"drafts",
"close",
]
def test_explicit_password_login_drives_login_page_then_returns_to_apply(
monkeypatch,
) -> None:
calls: list[str] = []
class Page:
def goto(self, url, **_kwargs) -> None:
calls.append(f"goto:{url}")
def wait_for_timeout(self, _ms: int) -> None:
return None
monkeypatch.setattr(
baibu, "_login_if_needed", lambda *_args, **_kwargs: calls.append("fill-login") or True
)
monkeypatch.setattr(
baibu,
"_wait_for_authenticated",
lambda *_args, **_kwargs: calls.append("authenticated"),
)
entry = _entry()
baibu._explicit_password_login(entry, Page(), manual_login=False)
# The apply URL is re-entered first so its redirect chain can surface the
# seller login page; only a missing form there falls back to login.taobao.
assert calls == [
f"goto:{entry.apply_url}",
"fill-login",
f"goto:{entry.apply_url}",
"authenticated",
]
class _FakeLock:
def __init__(self, manager: "_FakeLockManager", resource: str) -> None:
self._manager = manager
self._resource = resource
def __enter__(self):
self._manager.held.add(self._resource)
self._manager.entered.append(self._resource)
return self
def __exit__(self, *_args) -> None:
self._manager.held.discard(self._resource)
class _FakeLockManager:
def __init__(self, lock_root) -> None:
self.lock_root = lock_root
self.held: set[str] = set()
self.entered: list[str] = []
self.requested: list[tuple[str, int]] = []
def acquire(self, resource: str, *, owner: str, timeout_seconds: int):
self.requested.append((resource, timeout_seconds))
if getattr(self, "busy", False):
raise baibu.ResourceBusyError(resource)
return _FakeLock(self, resource)
def test_apply_entry_holds_account_lock_for_whole_session(monkeypatch) -> None:
calls: list[str] = []
lock_manager = _FakeLockManager(lock_root="locks-dir")
class FakeSettings:
data_root = Path("data-root")
@classmethod
def from_env(cls):
return cls()
monkeypatch.setattr(baibu, "Settings", FakeSettings)
monkeypatch.setattr(baibu, "LockManager", lambda lock_root: lock_manager)
monkeypatch.setattr(baibu, "_start_browser", lambda: _browser_spy(calls))
monkeypatch.setattr(baibu, "_login_if_needed", lambda *_args, **_kwargs: False)
monkeypatch.setattr(baibu, "_wait_for_authenticated", lambda *_args, **_kwargs: None)
monkeypatch.setattr(
baibu,
"_persist_account_state",
lambda *_args, **_kwargs: True,
)
monkeypatch.setattr(baibu, "_open_batch_import", lambda _page, _entry: None)
def assert_lock_held_during_upload(_page, _template):
assert lock_manager.held == {"browser-account:tmall-shop"}
calls.append("upload")
monkeypatch.setattr(baibu, "_upload_template", assert_lock_held_during_upload)
monkeypatch.setattr(baibu, "_wait_import_result", lambda *_args, **_kwargs: None)
monkeypatch.setattr(baibu, "_complete_draft_products", lambda *_args: None)
monkeypatch.setattr(baibu.time, "sleep", lambda _seconds: None)
baibu._apply_entry(_entry(), Path("template.xlsx"))
assert lock_manager.requested == [("browser-account:tmall-shop", 120)]
assert lock_manager.entered == ["browser-account:tmall-shop"]
assert lock_manager.held == set()
def test_apply_entry_reports_busy_account_lock(monkeypatch) -> None:
lock_manager = _FakeLockManager(lock_root="locks-dir")
lock_manager.busy = True
class FakeSettings:
data_root = Path("data-root")
@classmethod
def from_env(cls):
return cls()
monkeypatch.setattr(baibu, "Settings", FakeSettings)
monkeypatch.setattr(baibu, "LockManager", lambda lock_root: lock_manager)
started: list[str] = []
monkeypatch.setattr(
baibu, "_start_browser", lambda: started.append("start") or object()
)
with pytest.raises(baibu.BaibuApplyError, match="正被保活/登录占用"):
baibu._apply_entry(_entry(), Path("template.xlsx"))
assert started == []
def test_explicit_password_login_falls_back_to_plain_login_page(
monkeypatch,
) -> None:
calls: list[str] = []
class Page:
def goto(self, url, **_kwargs) -> None:
calls.append(f"goto:{url}")
def wait_for_timeout(self, _ms: int) -> None:
return None
monkeypatch.setattr(baibu, "_login_if_needed", lambda *_args, **_kwargs: False)
monkeypatch.setattr(
baibu,
"_wait_for_authenticated",
lambda *_args, **_kwargs: calls.append("authenticated"),
)
entry = _entry()
baibu._explicit_password_login(entry, Page(), manual_login=False)
assert calls == [
f"goto:{entry.apply_url}",
f"goto:{baibu._LOGIN_PAGE_URL}",
f"goto:{entry.apply_url}",
"authenticated",
]
def test_upload_template_supports_tmall_dialog_import_button( def test_upload_template_supports_tmall_dialog_import_button(
monkeypatch, monkeypatch,
tmp_path: Path, tmp_path: Path,
@@ -558,6 +905,32 @@ def test_wait_import_result_accepts_new_failure_summary_as_submitted(
baibu._wait_import_result(SimpleNamespace(), _entry()) baibu._wait_import_result(SimpleNamespace(), _entry())
def test_wait_import_result_accepts_slow_processing_after_acknowledgement(
monkeypatch,
capsys,
) -> None:
# Tmall acknowledged the batch but never returned a row-level summary
# within the acceptance window; the acknowledged submission still counts.
processing_text = (
"您于2026-10-14 14:51:00执行了商品批量导入操作,正在处理中。"
)
clock = {"now": 0.0}
def advance_monotonic() -> float:
clock["now"] += 15.0
return clock["now"]
monkeypatch.setattr(baibu, "_page_text", lambda _page: processing_text)
monkeypatch.setattr(baibu, "_refresh_batch_results", lambda _page: True)
monkeypatch.setattr(baibu.time, "sleep", lambda _seconds: None)
monkeypatch.setattr(baibu.time, "monotonic", advance_monotonic)
monkeypatch.setattr(baibu, "_RESULT_REFRESH_INTERVAL_SECONDS", 0)
baibu._wait_import_result(SimpleNamespace(), _entry())
assert "按导入提交成功" in capsys.readouterr().out
def test_wait_import_result_ignores_stale_summary_until_signature_changes( def test_wait_import_result_ignores_stale_summary_until_signature_changes(
monkeypatch, monkeypatch,
) -> None: ) -> None:
@@ -663,6 +1036,47 @@ def test_complete_draft_products_accepts_empty_data_page(monkeypatch) -> None:
baibu._complete_draft_products(page, entry) baibu._complete_draft_products(page, entry)
def test_complete_draft_products_retries_draft_tab_during_hydration(
monkeypatch,
) -> None:
entry = replace(_entry(), complete_drafts_after_import=True)
page = SimpleNamespace()
attempts = {"n": 0}
def flaky_tab(_page):
attempts["n"] += 1
return attempts["n"] >= 3
monkeypatch.setattr(baibu, "_click_draft_tab", flaky_tab)
monkeypatch.setattr(baibu, "_page_text", lambda _page: "加载中")
monkeypatch.setattr(baibu, "_has_visible_text", lambda _page, _labels: False)
monkeypatch.setattr(baibu, "_wait_for_draft_state", lambda _page, _entry: (0, 0))
monkeypatch.setattr(baibu.time, "sleep", lambda _seconds: None)
ticks = iter(range(10_000))
monkeypatch.setattr(baibu.time, "monotonic", lambda: next(ticks) * 0.4)
baibu._complete_draft_products(page, entry)
assert attempts["n"] == 3
def test_complete_draft_products_reports_missing_draft_entry_after_retries(
monkeypatch,
) -> None:
entry = replace(_entry(), complete_drafts_after_import=True)
page = SimpleNamespace()
monkeypatch.setattr(baibu, "_click_draft_tab", lambda _page: False)
monkeypatch.setattr(baibu, "_page_text", lambda _page: "加载中")
monkeypatch.setattr(baibu, "_has_visible_text", lambda _page, _labels: False)
monkeypatch.setattr(baibu.time, "sleep", lambda _seconds: None)
ticks = iter(range(10_000))
monkeypatch.setattr(baibu.time, "monotonic", lambda: next(ticks) * 0.4)
with pytest.raises(baibu.BaibuApplyError, match="未找到草稿页入口"):
baibu._complete_draft_products(page, entry)
def test_return_to_draft_list_falls_back_to_browser_history(monkeypatch) -> None: def test_return_to_draft_list_falls_back_to_browser_history(monkeypatch) -> None:
events: list[str] = [] events: list[str] = []
+1 -1
View File
@@ -28,7 +28,7 @@ from gyxx_flow.workflow.engine import WorkflowEngine
from gyxx_flow.workflow.model import StepDefinition, WorkflowDefinition from gyxx_flow.workflow.model import StepDefinition, WorkflowDefinition
from gyxx_flow.workflow.steps import StepExecution from gyxx_flow.workflow.steps import StepExecution
_NOW = datetime(2026, 10, 8, 12, 0, 0, tzinfo=timezone.utc) _NOW = datetime.now(timezone.utc).replace(microsecond=0)
def _settings(tmp_path: Path) -> Settings: def _settings(tmp_path: Path) -> Settings:
-1
View File
@@ -36,7 +36,6 @@ def test_repository_keeps_the_simplified_public_structure() -> None:
"note-inventory.md", "note-inventory.md",
"plan.md", "plan.md",
"runbook.md", "runbook.md",
"workbench.md",
"workflow-acceptance-test-report.md", "workflow-acceptance-test-report.md",
"workflow-acceptance-test-requirements.md", "workflow-acceptance-test-requirements.md",
"workflow-console.md", "workflow-console.md",
+359
View File
@@ -0,0 +1,359 @@
from __future__ import annotations
import json
from datetime import date, datetime, timezone
from pathlib import Path
from typing import Any
import pytest
from gyxx_flow.ops.audit import (
SCHEMA_SQL,
AuditStore,
split_step_key,
summarize_cookies,
)
from gyxx_flow.ops.index import RunIndex
DOUYIN_COOKIES: list[dict[str, Any]] = [
{
"name": "sessionid",
"value": "AAA",
"domain": ".jinritemai.com",
"expires": 1766000000.0,
},
{
"name": "sid_tt",
"value": "BBB",
"domain": ".jinritemai.com",
"expires": 1766100000.0,
},
{
"name": "unrelated",
"value": "CCC",
"domain": ".example.com",
"expires": 1700000000.0,
},
{
"name": "session",
"value": "DDD",
"domain": ".jinritemai.com",
"expires": -1,
},
]
class _FakeCursor:
def __init__(self, log: list[Any], rows: list[dict[str, Any]]) -> None:
self._log = log
self._rows = rows
def __enter__(self) -> "_FakeCursor":
return self
def __exit__(self, *_: object) -> bool:
return False
def execute(self, sql: str, params: Any = None) -> None:
self._log.append(("execute", sql, params))
def executemany(self, sql: str, rows: Any) -> None:
self._log.append(("executemany", sql, list(rows)))
def fetchall(self) -> list[dict[str, Any]]:
return self._rows
def fetchone(self) -> dict[str, Any] | None:
# 默认模拟「ops_audit_schema 尚不存在」,从而走首次建表分支。
return None
def close(self) -> None:
return None
class _FakeConnection:
def __init__(self, log: list[Any], rows: list[dict[str, Any]] | None = None) -> None:
self._log = log
self._rows = rows or []
def cursor(self) -> _FakeCursor:
return _FakeCursor(self._log, self._rows)
def commit(self) -> None:
self._log.append(("commit",))
def close(self) -> None:
self._log.append(("close",))
def _store_with_fake(
tmp_path: Path,
*,
environment: dict[str, str] | None = None,
rows: list[dict[str, Any]] | None = None,
) -> tuple[AuditStore, list[Any]]:
log: list[Any] = []
store = AuditStore(
tmp_path,
environment=environment or {"GYXX_POSTGRES_DSN": "postgresql://fake/audit"},
connect=lambda: _FakeConnection(log, rows),
)
return store, log
# ----------------------------------------------------------------------
# cookie 快照
# ----------------------------------------------------------------------
def test_summarize_cookies_keeps_only_required_cookies() -> None:
snapshot = summarize_cookies(
DOUYIN_COOKIES,
required_domains=("jinritemai.com",),
required_names=("sessionid", "sid_tt", "session"),
)
assert snapshot.cookie_count == 4
assert snapshot.required_count == 3
# 最早的必需 cookie 过期时间:1766000000,而不是无关 cookie 的 1700000000
assert snapshot.declared_expires_at == datetime.fromtimestamp(
1766000000.0, tz=timezone.utc
)
def test_summarize_cookies_ignores_session_cookies_for_expiry() -> None:
snapshot = summarize_cookies(
DOUYIN_COOKIES,
required_domains=("jinritemai.com",),
required_names=("session",),
)
assert snapshot.required_count == 1
assert snapshot.declared_expires_at is None
def test_summarize_cookies_fingerprint_tracks_value_changes_only() -> None:
baseline = summarize_cookies(DOUYIN_COOKIES, required_domains=("jinritemai.com",))
reordered = summarize_cookies(
list(reversed(DOUYIN_COOKIES)), required_domains=("jinritemai.com",)
)
replaced = [dict(cookie) for cookie in DOUYIN_COOKIES]
replaced[0]["value"] = "ROTATED"
assert baseline.fingerprint # 非空
assert baseline.fingerprint == reordered.fingerprint
assert baseline.fingerprint != summarize_cookies(
replaced, required_domains=("jinritemai.com",)
).fingerprint
def test_summarize_cookies_without_requirements_counts_everything() -> None:
snapshot = summarize_cookies(DOUYIN_COOKIES)
assert snapshot.required_count == 4
assert snapshot.declared_expires_at == datetime.fromtimestamp(
1700000000.0, tz=timezone.utc
)
def test_summarize_cookies_has_no_plaintext_value_in_fingerprint() -> None:
snapshot = summarize_cookies(DOUYIN_COOKIES, required_domains=("jinritemai.com",))
assert "AAA" not in snapshot.fingerprint
assert len(snapshot.fingerprint) == 32
def test_split_step_key_handles_retry_suffix() -> None:
assert split_step_key("sync.attempt-2") == ("sync", 2)
assert split_step_key("collect_collaborators") == ("collect_collaborators", 1)
assert split_step_key("a.attempt-x") == ("a.attempt-x", 1)
# ----------------------------------------------------------------------
# 写入装配
# ----------------------------------------------------------------------
def test_record_run_upserts_run_and_steps(tmp_path: Path) -> None:
store, log = _store_with_fake(tmp_path)
ok = store.record_run(
{
"workflow_id": "content.metrics.daily",
"run_id": "content.metrics.daily__20261018__abc",
"business_date": "2026-10-18",
"status": "success",
"mode": "dry_run",
"shadow": False,
"started_at": "2026-10-19T01:41:43+00:00",
"ended_at": "2026-10-19T01:41:48+00:00",
"error": None,
"journal_path": "runs/content.metrics.daily/run.json",
},
{
"sync.attempt-1": {
"status": "success",
"exit_code": 0,
"started_at": "2026-10-19T01:41:44+00:00",
"ended_at": "2026-10-19T01:41:46+00:00",
},
"collect.attempt-1": {"status": "failed", "exit_code": 4, "error": "boom"},
},
)
assert ok is True
statements = [entry for entry in log if entry[0] == "execute"]
assert any("CREATE TABLE IF NOT EXISTS ops_workflow_runs" in item[1] for item in statements)
insert = next(item for item in statements if "INSERT INTO ops_workflow_runs" in item[1])
params = insert[2]
assert params["business_date"] == date(2026, 10, 18)
assert params["duration_seconds"] == 5
assert params["step_total"] == 2
assert params["step_success"] == 1
assert params["step_failed"] == 1
batch = next(item for item in log if item[0] == "executemany")
rows = {item["step_id"]: item for item in batch[2]}
assert rows["sync"]["attempt"] == 1
assert rows["sync"]["duration_seconds"] == 2
assert rows["collect"]["exit_code"] == 4
assert rows["collect"]["error"] == "boom"
def test_record_run_rejects_incomplete_payload(tmp_path: Path) -> None:
store, log = _store_with_fake(tmp_path)
assert store.record_run({"workflow_id": "x"}) is False
assert not [entry for entry in log if entry[0] == "execute"]
def test_record_cookie_event_writes_value_free_snapshot(tmp_path: Path) -> None:
store, log = _store_with_fake(tmp_path)
snapshot = summarize_cookies(DOUYIN_COOKIES, required_domains=("jinritemai.com",))
ok = store.record_cookie_event(
account_id="douyin-shop",
state="healthy",
snapshot=snapshot,
probe_status="success",
probe_source="keepalive",
workflow_id="content.metrics.daily",
run_id="run-1",
)
assert ok is True
insert = next(
item
for item in log
if item[0] == "execute" and "INSERT INTO ops_account_cookie_events" in item[1]
)
params = insert[2]
assert params["account_id"] == "douyin-shop"
assert params["state"] == "healthy"
assert params["fingerprint"] == snapshot.fingerprint
assert params["declared_expires_at"] == snapshot.declared_expires_at
# 明文值不得出现在任何参数里
assert "AAA" not in json.dumps(params, default=str)
def test_record_cookie_event_rejects_unknown_state(tmp_path: Path) -> None:
store, _ = _store_with_fake(tmp_path)
with pytest.raises(ValueError):
store.record_cookie_event(account_id="a", state="definitely-not-valid")
# ----------------------------------------------------------------------
# 降级与熔断
# ----------------------------------------------------------------------
def test_missing_dsn_never_raises_and_logs_failure(tmp_path: Path) -> None:
store = AuditStore(tmp_path, environment={})
assert store.record_run(
{
"workflow_id": "content.metrics.daily",
"run_id": "r1",
"business_date": "2026-10-18",
"status": "success",
}
) is False
assert "审计数据库配置缺失" in store.last_error
failure_log = tmp_path / "logs" / "ops-audit" / "audit-failures.log"
assert failure_log.exists()
assert json.loads(failure_log.read_text(encoding="utf-8").splitlines()[0])["operation"] == (
"record_run"
)
def test_audit_can_be_disabled_without_touching_the_database(tmp_path: Path) -> None:
store, log = _store_with_fake(
tmp_path, environment={"GYXX_POSTGRES_DSN": "postgresql://fake", "GYXX_OPS_AUDIT": "off"}
)
assert store.enabled is False
assert store.record_cookie_event(account_id="a", state="healthy") is False
assert log == []
def test_consecutive_failures_open_the_circuit(tmp_path: Path) -> None:
attempts = 0
def _boom() -> Any:
nonlocal attempts
attempts += 1
raise OSError("database unreachable")
store = AuditStore(
tmp_path,
environment={"GYXX_POSTGRES_DSN": "postgresql://fake", "GYXX_OPS_AUDIT_FAILURE_COOLDOWN": "60"},
connect=_boom,
)
payload = {
"workflow_id": "w",
"run_id": "r",
"business_date": "2026-10-18",
"status": "success",
}
for _ in range(3):
store.record_run(payload)
assert attempts == 3
# 熔断生效后不再尝试连接,避免每个工作流都卡在 TCP 超时上
assert store.record_run(payload) is False
assert attempts == 3
# ----------------------------------------------------------------------
# 与运行索引的集成
# ----------------------------------------------------------------------
def test_run_index_still_works_when_audit_is_unavailable(tmp_path: Path) -> None:
from gyxx_flow.core.context import RunContext
from gyxx_flow.core.layout import DataLayout
from gyxx_flow.core.records import RunJournal
layout = DataLayout(tmp_path)
context = RunContext.create("content.metrics.daily", "2026-10-18")
journal = RunJournal.create(layout, context, mode="dry_run")
journal.finalize("success")
record = RunIndex(tmp_path).index_journal(journal)
assert record.workflow_id == "content.metrics.daily"
assert record.status == "success"
# 运行索引文件仍然写出,审计失败不影响主链路
assert (tmp_path / "state" / "ops" / "run-index" / f"{record.run_id}.json").exists()
def test_schema_sql_defines_both_audit_surfaces() -> None:
assert "ops_workflow_runs" in SCHEMA_SQL
assert "ops_workflow_run_steps" in SCHEMA_SQL
assert "ops_account_cookie_events" in SCHEMA_SQL
assert "ops_account_cookie_lifetimes" in SCHEMA_SQL
assert "ops_account_cookie_stats" in SCHEMA_SQL
+3 -3
View File
@@ -102,10 +102,10 @@ def test_project_registry_covers_every_script_with_unique_stable_port() -> None:
data_root=project_root / "var", data_root=project_root / "var",
) )
assert len(scripts.command_ids) == 60 assert len(scripts.command_ids) == 61
assert len(first.command_ids) == 157 assert len(first.command_ids) == 158
bindings = [first.binding_for(command_id) for command_id in first.command_ids] bindings = [first.binding_for(command_id) for command_id in first.command_ids]
assert len({binding.cdp_port for binding in bindings}) == 157 assert len({binding.cdp_port for binding in bindings}) == 158
assert all(22000 <= binding.cdp_port <= 22999 for binding in bindings) assert all(22000 <= binding.cdp_port <= 22999 for binding in bindings)
assert all(binding.cdp_url.startswith("http://127.0.0.1:") for binding in bindings) assert all(binding.cdp_url.startswith("http://127.0.0.1:") for binding in bindings)
assert bindings == [second.binding_for(item.script_id) for item in bindings] assert bindings == [second.binding_for(item.script_id) for item in bindings]
+2 -2
View File
@@ -83,7 +83,7 @@ def test_default_registry_covers_every_executable_workflow_entry() -> None:
assert scheduled_workflows == 39 assert scheduled_workflows == 39
assert manual_workflows == 0 assert manual_workflows == 0
assert len(workflow_entries) == 55 assert len(workflow_entries) == 55
assert len(catalog.scripts) == 60 assert len(catalog.scripts) == 61
assert workflow_entries < set(catalog.script_ids) assert workflow_entries < set(catalog.script_ids)
assert { assert {
"content.mapping.rebuild", "content.mapping.rebuild",
@@ -280,7 +280,7 @@ def test_cli_lists_and_dry_runs_registered_commands(tmp_path: Path) -> None:
assert main(["scripts", "list", "--json"], settings=settings, stdout=output) == 0 assert main(["scripts", "list", "--json"], settings=settings, stdout=output) == 0
rows = json.loads(output.getvalue()) rows = json.loads(output.getvalue())
assert len(rows) == 60 assert len(rows) == 61
assert {row["module"] for row in rows} == { assert {row["module"] for row in rows} == {
"content_marketing", "content_marketing",
"product_commerce", "product_commerce",
-139
View File
@@ -1,139 +0,0 @@
# GYXX 智能工作台(deepseek-harness 集成)
以 [deepseek-harness](https://github.com/deepseek-ai/deepseek-harness)(`dsh`,DeepSeek 智能体底座)
为 agent 运行时,把 gyxx-flow 的现有工作流**以插件形式**扩展进 dsh Web UI,形成一个
可监控、可启动、可智能诊断/修复工作流的智能工作台。
```
┌──────────────────────── dsh Web UI(DeepSeek 智能体)────────────────────────┐
│ 侧边栏「工作流」面板 ───────────────┐ │
│ (插件客户端包 shell.overlay 抽屉) │ 对话:选中工作流后提问 / 诊断 / 修复 │
└─────────┬────────────────────────────┴───────────────────▲─────────────────┘
│ 本机回环桥接 127.0.0.1:8790 │ 7 个工作流工具
┌─────────▼─────────────────────────────────────────────────┴─────────────────┐
│ gyxx-workbench 宿主插件(workbench/plugin/gyxx-workbench.mjs) │
│ 工具注册 · 系统提示词 · 失败监控器 · 桥接服务 · 会话创建 │
└─────────┬────────────────────────────────────────────────────────────────────┘
│ HTTP(只读 + 受控写)
┌─────────▼─────────────────────────────────────────┐
│ gyxx console(127.0.0.1:8765,gyxx-flow 现有控制台) │
│ /api/overview · /api/workflows/* · /api/dynamic-configs │
└─────────┬─────────────────────────────────────────┘
┌─────────▼─────────────────────────────────────────┐
│ gyxx-flow 工作流引擎(LangGraph)· 调度器 · RunJournal │
└───────────────────────────────────────────────────┘
```
## 快速开始
前置条件:Python 3.12 + `uv sync` 已完成;Node.js 22.19+;一个 DeepSeek API Key
(`DEEPSEEK_API_KEY`,dsh 自身要求)。
```powershell
# Windows:渲染补丁、拉起控制台(注入云端凭据)、启动 dsh Web UI
powershell -File workbench\bin\start-workbench.ps1
# 控制台凭据文件不是默认路径时
powershell -File workbench\bin\start-workbench.ps1 -ConsoleEnvFile <你的.env 路径>
```
```bash
# Linux/macOS
bash workbench/bin/start-workbench.sh [console_env_file]
```
启动后打开 dsh Web UI(默认 <http://127.0.0.1:3080>):
- 侧边栏底部出现「工作流」按钮 → 打开工作流面板;
- 面板按模块分组列出全部调度工作流(状态点:绿=成功 / 红=失败 / 蓝=运行中 / 灰=未运行);
- **选中任意工作流**后可直接:
- 「提问」:带着该工作流上下文创建智能体会话,自由提问;
- 「诊断」:自动获取最近一次失败运行的诊断包(步骤、脱敏日志)并输出根因报告;
- 「修复」:智能体先诊断再给修复方案,**任何正式执行/调度修改必须先经你确认**;
- 「试运行 / 正式运行 / 停止」:对应控制台的受控执行语义(试运行无外部副作用)。
降级方案:若 dsh 客户端包因版本差异未能加载,直接打开桥接服务自带的独立面板
<http://127.0.0.1:8790/>,功能与侧边栏面板一致(零依赖页面)。
## 目录结构
```
workbench/
cordis.template.yml # dsh 组合补丁模板(启动脚本渲染出 cordis.local.yml)
bin/
start-workbench.ps1 # Windows 一键启动
start-workbench.sh # Linux/macOS 一键启动
plugin/ # dsh 插件包(@gyxx/dsh-plugin-gyxx-workbench)
package.json # 含 dsh.client 声明(浏览器包发现契约)
gyxx-workbench.mjs # 宿主插件:工具 / 提示词 / 监控 / 桥接(零运行时依赖)
client/
src/ # 侧边栏面板源码(React,打包时 react 外置)
standalone.html # 降级独立面板(桥接服务直接托管)
scripts/build-client.mjs
lib/client.js # 已构建的浏览器包(随仓库提交,改源码后需重建)
tests/ # node --test 冒烟测试
```
## 智能体工具清单
| 工具 | 说明 | 副作用 |
| --- | --- | --- |
| `gyxx_workflow_list` | 全部工作流及调度、最近运行状态 | 无 |
| `gyxx_workflow_detail` | 单工作流定义(步骤/依赖/重放策略/调度) | 无 |
| `gyxx_workflow_runs` | 最近运行历史(步骤级状态、退出码、脱敏错误) | 无 |
| `gyxx_workflow_diagnose` | 一次运行的完整诊断包(journal 路径 + 脱敏日志尾部) | 无 |
| `gyxx_workflow_trigger` | 触发运行;默认试运行 | 试运行无副作用;正式执行需 `execute=true` + `confirmed=true` |
| `gyxx_workflow_cancel` | 停止手动/定时运行 | 有(停止进程) |
| `gyxx_schedule_update` | 启用/停用/改调度(读-改-写,带版本校验) | 有(改 `config/schedules.json`) |
## 安全模型
- 宿主插件与控制台都只绑定 `127.0.0.1`;桥接服务的写操作要求 `x-gyxx-workbench: 1`
自定义头 + `application/json`,拒绝跨站表单提交;CORS 仅回环来源。
- 控制台自身的守卫不变:正式执行仍需 `confirmed=true`,写操作仍需
`X-GYXX-Console: 1` 与同源检查;令牌模式(`GYXX_CONSOLE_TOKEN`)对插件同样生效。
- 系统提示词固化「诊断 → 方案 → 用户确认 → 执行」的修复顺序,禁止智能体跳过确认。
- 日志经控制台脱敏管道(口令/Token/URL 凭据打码)后才进入对话上下文。
## 失败监控
宿主插件每 30s(`pollIntervalMs`)轮询 `/api/overview`:某工作流出现**新的**失败运行时
→ 面板右下角弹出告警 toast(可一键「立即诊断」);`autoDiagnose: true` 时还会自动创建
诊断会话。告警去重以 `workflow_id + run_id` 为准,恢复成功后重置。
## 配置项(cordis.local.yml → config)
| 键 | 默认 | 说明 |
| --- | --- | --- |
| `consoleBaseUrl` | `http://127.0.0.1:8765`(或环境变量 `GYXX_CONSOLE_URL`) | gyxx 控制台地址 |
| `consoleToken` | 环境变量 `GYXX_CONSOLE_TOKEN` | 控制台访问令牌(控制台以令牌模式运行时必填) |
| `bridgeHost` / `bridgePort` | `127.0.0.1` / `8790` | 桥接服务监听地址 |
| `pollIntervalMs` | `30000` | 失败监控轮询间隔 |
| `autoDiagnose` | `false` | 发现失败时自动创建诊断会话 |
| `projectRoot` | dsh 进程 cwd | 新建智能体会话的工作目录 |
## 开发
```bash
# 重建客户端包(修改 client/src 后必须执行并提交 lib/client.js)
cd workbench/plugin && npm install && npm run build
# 宿主插件冒烟测试(不需要 dsh / gyxx console)
cd workbench/plugin && npm test
# Python 侧端点测试
uv run pytest tests/test_console.py -k "workflow_detail or run_detail or diagnosis"
```
## 依赖的 dsh 扩展点(上游契约)
- `ctx.tools.register()` 原始 JSON-Schema 工具定义(cookbook: extension-cookbook)
- `ctx.systemPrompt.section()` 系统提示词段
- cordis.yml `--patch` 组合覆盖(`apps/cli/src/args.ts`,npm 版同样支持)
- `dsh.client` package.json 声明 → 客户端包发现(`packages/client/modules`)
- 插槽:`sidebar.footer.action`(list,追加)、`shell.overlay`(list,追加)
- `ctx.agents.create()` + `agent.followup()` 编程式会话
dsh 处于 developer preview,扩展点可能变化;升级 dsh 后若面板消失,先检查
浏览器控制台模块加载错误,再核对上述插槽名是否仍存在于
`packages/client/ui-layout` / `ui-sidebar` 的 SlotMap。
-77
View File
@@ -1,77 +0,0 @@
# GYXX 智能工作台一键启动(Windows PowerShell)
# 用法:
# powershell -File workbench\bin\start-workbench.ps1
# powershell -File workbench\bin\start-workbench.ps1 -ConsoleEnvFile D:\product-collector-analyze-flow\.env
#
# 行为:
# 1. 渲染 workbench/cordis.local.yml(插件绝对路径)
# 2. 若 8765 控制台未运行,则以 --env-file 注入云端凭据启动 gyxx console
# 3. 启动 dsh Web UI 并应用工作台补丁(优先 $env:DSH_HOME 源码目录,其次 npx @deepseek-ai/dsh)
[CmdletBinding()]
param(
[string]$ConsoleEnvFile = 'D:\product-collector-analyze-flow\.env',
[int]$ConsolePort = 8765,
[int]$BridgePort = 8790,
[switch]$SkipConsole
)
$ErrorActionPreference = 'Stop'
$RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..')).Path
$WorkbenchDir = Join-Path $RepoRoot 'workbench'
$LocalYml = Join-Path $WorkbenchDir 'cordis.local.yml'
# 1) 渲染 cordis.local.yml
$template = Get-Content (Join-Path $WorkbenchDir 'cordis.template.yml') -Raw -Encoding UTF8
$workbenchDirPosix = ($WorkbenchDir -replace '\\', '/')
$projectRootPosix = ($RepoRoot -replace '\\', '/')
# Windows 上 dsh 的 ESM 加载器要求 file:// URL
$workbenchModuleUrl = 'file:///' + $workbenchDirPosix
$rendered = $template.Replace('__WORKBENCH_DIR__', $workbenchModuleUrl).Replace('__PROJECT_ROOT__', $projectRootPosix)
$rendered = $rendered -replace 'bridgePort: 8790', "bridgePort: $BridgePort"
$rendered = $rendered -replace "consoleBaseUrl: 'http://127.0.0.1:8765'", "consoleBaseUrl: 'http://127.0.0.1:$ConsolePort'"
[System.IO.File]::WriteAllText($LocalYml, $rendered, (New-Object System.Text.UTF8Encoding($false)))
Write-Host "[workbench] 已生成 $LocalYml"
# 2) 确保 gyxx console 在运行
if (-not $SkipConsole) {
$consoleUp = $false
try {
$null = Invoke-WebRequest -UseBasicParsing -Uri "http://127.0.0.1:$ConsolePort/api/overview" -TimeoutSec 3
$consoleUp = $true
} catch { $consoleUp = $false }
if ($consoleUp) {
Write-Host "[workbench] gyxx console 已在 http://127.0.0.1:$ConsolePort 运行"
} else {
if (-not (Test-Path $ConsoleEnvFile)) {
Write-Warning "缺少 $ConsoleEnvFile —— 正式执行将无法注入云端凭据(AGENTS.md 约定)。"
}
$consoleArgs = @('run', 'gyxx', 'console', '--port', "$ConsolePort", '--env-file', $ConsoleEnvFile)
Write-Host "[workbench] 启动 gyxx console: uv $($consoleArgs -join ' ')"
Start-Process -FilePath 'uv' -ArgumentList $consoleArgs -WorkingDirectory $RepoRoot -WindowStyle Minimized
$deadline = (Get-Date).AddSeconds(30)
do {
Start-Sleep -Milliseconds 800
try {
$null = Invoke-WebRequest -UseBasicParsing -Uri "http://127.0.0.1:$ConsolePort/api/overview" -TimeoutSec 2
$consoleUp = $true
} catch { $consoleUp = $false }
} until ($consoleUp -or (Get-Date) -gt $deadline)
if (-not $consoleUp) { Write-Warning 'gyxx console 启动超时,工作台仍可启动但工具会提示控制台离线。' }
}
}
# 3) 启动 dsh Web UI
if ($env:DSH_HOME -and (Test-Path (Join-Path $env:DSH_HOME 'package.json'))) {
Write-Host "[workbench] 使用源码版 dsh: $($env:DSH_HOME)"
Push-Location $env:DSH_HOME
try {
& pnpm dsh web --patch $LocalYml
} finally {
Pop-Location
}
} else {
Write-Host '[workbench] 使用 npm 版 dsh(npx @deepseek-ai/dsh)'
& npx -y '@deepseek-ai/dsh' web --patch $LocalYml
}
-43
View File
@@ -1,43 +0,0 @@
#!/usr/bin/env bash
# GYXX 智能工作台一键启动(Linux/macOS)
# 用法:bash workbench/bin/start-workbench.sh [console_env_file]
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
WORKBENCH_DIR="$REPO_ROOT/workbench"
LOCAL_YML="$WORKBENCH_DIR/cordis.local.yml"
CONSOLE_ENV_FILE="${1:-${GYXX_CONSOLE_ENV_FILE:-/etc/gyxx/flow.env}}"
CONSOLE_PORT="${GYXX_CONSOLE_PORT:-8765}"
BRIDGE_PORT="${GYXX_WB_BRIDGE_PORT:-8790}"
# 1) 渲染 cordis.local.yml(插件路径必须为绝对路径)
# dsh 的 ESM 加载器要求 file:// URL(Windows 上必须,POSIX 上同样可用)
sed \
-e "s|__WORKBENCH_DIR__|file://$WORKBENCH_DIR|g" \
-e "s|__PROJECT_ROOT__|$REPO_ROOT|g" \
-e "s|bridgePort: 8790|bridgePort: $BRIDGE_PORT|g" \
-e "s|http://127.0.0.1:8765|http://127.0.0.1:$CONSOLE_PORT|g" \
"$WORKBENCH_DIR/cordis.template.yml" > "$LOCAL_YML"
echo "[workbench] 已生成 $LOCAL_YML"
# 2) 确保 gyxx console 在运行
if ! curl -fsS -m 3 "http://127.0.0.1:$CONSOLE_PORT/api/overview" >/dev/null 2>&1; then
if [ ! -f "$CONSOLE_ENV_FILE" ]; then
echo "[workbench] 警告:缺少 $CONSOLE_ENV_FILE —— 正式执行将无法注入云端凭据(AGENTS.md 约定)。" >&2
fi
echo "[workbench] 启动 gyxx console(端口 $CONSOLE_PORT)"
(cd "$REPO_ROOT" && nohup uv run gyxx console --port "$CONSOLE_PORT" --env-file "$CONSOLE_ENV_FILE" >/dev/null 2>&1 &)
for _ in $(seq 1 30); do
sleep 1
if curl -fsS -m 2 "http://127.0.0.1:$CONSOLE_PORT/api/overview" >/dev/null 2>&1; then break; fi
done
fi
# 3) 启动 dsh Web UI
if [ -n "${DSH_HOME:-}" ] && [ -f "$DSH_HOME/package.json" ]; then
echo "[workbench] 使用源码版 dsh: $DSH_HOME"
(cd "$DSH_HOME" && pnpm dsh web --patch "$LOCAL_YML")
else
echo "[workbench] 使用 npm 版 dsh(npx @deepseek-ai/dsh)"
npx -y @deepseek-ai/dsh web --patch "$LOCAL_YML"
fi
-22
View File
@@ -1,22 +0,0 @@
# GYXX 智能工作台 —— dsh Web 组合补丁(模板)。
# 启动脚本会把 __WORKBENCH_DIR__ / __PROJECT_ROOT__ 渲染为绝对路径后生成
# cordis.local.yml,再通过 `dsh web --patch workbench/cordis.local.yml` 加载。
# 不要直接手工使用本文件:dsh 要求插件路径为绝对路径(Windows 上必须是 file:// URL)。
- insert:
- id: gyxx-workbench
name: '__WORKBENCH_DIR__/plugin/gyxx-workbench.mjs'
config:
# gyxx 控制台 HTTP API(gyxx console,默认 8765 端口)
consoleBaseUrl: 'http://127.0.0.1:8765'
# 访问令牌:缺省读取进程环境变量 GYXX_CONSOLE_TOKEN
# consoleToken: ''
# 侧边栏面板桥接服务(本机回环)
bridgeHost: '127.0.0.1'
bridgePort: 8790
# 失败监控轮询间隔(毫秒)
pollIntervalMs: 30000
# 发现失败时是否自动创建诊断会话(默认 false,只在面板弹告警)
autoDiagnose: false
# 智能体会话的工作目录(数据根目录下的日志可用文件工具直接查看)
projectRoot: '__PROJECT_ROOT__'
-355
View File
@@ -1,355 +0,0 @@
/** 工作流面板:侧边抽屉,列出全部工作流,选中后可提问 / 诊断 / 修复 / 启停。 */
import { useSyncExternalStore, useEffect, useCallback } from 'react'
import { getState, subscribe, update, pushToast } from './store.js'
import { api } from './api.js'
function useWorkbench() {
return useSyncExternalStore(subscribe, getState)
}
const STATUS_LABELS = {
success: '成功',
failed: '失败',
running: '运行中',
cancelled: '已停止',
none: '未运行',
}
function statusOf(workflow) {
if (workflow.active_run) return 'running'
return workflow.last_run?.status ?? 'none'
}
function formatTime(value) {
if (!value) return '—'
const date = new Date(value)
if (Number.isNaN(date.getTime())) return String(value)
const pad = (n) => String(n).padStart(2, '0')
return `${pad(date.getMonth() + 1)}-${pad(date.getDate())} ${pad(date.getHours())}:${pad(date.getMinutes())}`
}
function formatDuration(seconds) {
if (seconds == null) return ''
if (seconds < 60) return `${seconds}s`
return `${Math.floor(seconds / 60)}m${seconds % 60 ? `${seconds % 60}s` : ''}`
}
export async function refreshWorkflows() {
update({ loading: true, error: null })
try {
const payload = await api.listWorkflows()
update({
loading: false,
workflows: payload.workflows ?? [],
summary: payload.summary ?? null,
generatedAt: payload.generated_at ?? null,
})
} catch (error) {
update({ loading: false, error: error.message })
}
}
async function selectWorkflow(workflowId) {
update({ selectedId: workflowId, detailLoading: true, diagnosis: null, runs: [] })
try {
const [detail, runs] = await Promise.all([
api.workflowDetail(workflowId).catch(() => null),
api.workflowRuns(workflowId, 6).catch(() => ({ runs: [] })),
])
update({ detailLoading: false, detail, runs: runs.runs ?? [] })
} catch (error) {
update({ detailLoading: false, error: error.message })
}
}
async function loadDiagnosis(workflowId, runId) {
update({ diagnosisLoading: true })
try {
const payload = await api.diagnosis(workflowId, runId)
update({ diagnosisLoading: false, diagnosis: payload })
} catch (error) {
update({ diagnosisLoading: false })
pushToast({ kind: 'error', text: `诊断数据加载失败:${error.message}` })
}
}
async function doAsk(action) {
const s = getState()
if (!s.selectedId) return
const question = s.question.trim()
try {
const payload = await api.ask({
workflow_id: s.selectedId,
action,
question,
})
const label = { ask: '提问', diagnose: '诊断', repair: '修复' }[action] ?? '会话'
pushToast({
kind: 'ok',
text: `已创建${label}会话(${String(payload.session_id).slice(0, 18)}…),请在左侧会话列表中查看`,
})
update({ question: '' })
} catch (error) {
pushToast({ kind: 'error', text: `创建会话失败:${error.message}` })
}
}
async function doTrigger(execute) {
const s = getState()
if (!s.selectedId) return
if (execute) {
const ok = window.confirm(
`正式执行 ${s.selectedId}(业务日期 ${s.businessDate})会对外部系统产生真实写入。确认继续?`,
)
if (!ok) return
}
try {
await api.trigger({
workflow_id: s.selectedId,
business_date: s.businessDate,
execute,
confirmed: execute,
})
pushToast({ kind: 'ok', text: execute ? '已发起正式执行' : '已发起试运行(无外部副作用)' })
setTimeout(refreshWorkflows, 1500)
} catch (error) {
pushToast({ kind: 'error', text: `启动失败:${error.message}` })
}
}
async function doCancel() {
const s = getState()
const active = s.detail?.workflow?.active_run
if (!s.selectedId) return
try {
if (active?.operation_id) {
await api.cancel({ workflow_id: s.selectedId, operation_id: active.operation_id })
} else {
await api.cancel({ workflow_id: s.selectedId, scheduled: true })
}
pushToast({ kind: 'ok', text: '已发送停止指令' })
setTimeout(refreshWorkflows, 1500)
} catch (error) {
pushToast({ kind: 'error', text: `停止失败:${error.message}` })
}
}
function SummaryPills({ summary, workflows }) {
const failed = workflows.filter((w) => statusOf(w) === 'failed').length
const running = workflows.filter((w) => w.active_run).length
return (
<div className="gyxxwb-pills">
<span className="gyxxwb-pill">共 {summary?.total ?? workflows.length}</span>
{running > 0 && <span className="gyxxwb-pill gyxxwb-pill-running">运行中 {running}</span>}
{failed > 0 && <span className="gyxxwb-pill gyxxwb-pill-failed">失败 {failed}</span>}
</div>
)
}
function WorkflowRow({ workflow, selected, onSelect }) {
const status = statusOf(workflow)
return (
<button
type="button"
className={`gyxxwb-row ${selected ? 'gyxxwb-row-selected' : ''}`}
onClick={() => onSelect(workflow.id)}
>
<span className={`gyxxwb-dot gyxxwb-dot-${status}`} />
<span className="gyxxwb-row-main">
<span className="gyxxwb-row-name">{workflow.name}</span>
<span className="gyxxwb-row-id">{workflow.id}</span>
</span>
<span className="gyxxwb-row-meta">
{workflow.last_run ? formatTime(workflow.last_run.ended_at ?? workflow.last_run.started_at) : '—'}
</span>
</button>
)
}
function RunList({ runs, onDiagnose }) {
if (!runs.length) return <div className="gyxxwb-empty">暂无运行记录</div>
return (
<div className="gyxxwb-runs">
{runs.map((run) => (
<button
type="button"
key={run.run_id}
className="gyxxwb-run"
onClick={() => onDiagnose(run.run_id)}
title="点击查看诊断详情"
>
<span className={`gyxxwb-dot gyxxwb-dot-${run.status}`} />
<span>{run.business_date}</span>
<span className="gyxxwb-run-mode">{run.mode === 'execute' ? '正式' : '试运行'}</span>
<span>{formatDuration(run.duration_seconds)}</span>
{run.error && <span className="gyxxwb-run-error">{run.error.slice(0, 60)}</span>}
</button>
))}
</div>
)
}
function DiagnosisView({ diagnosis, loading }) {
if (loading) return <div className="gyxxwb-empty">正在加载诊断数据…</div>
if (!diagnosis) return null
if (diagnosis.message) return <div className="gyxxwb-empty">{diagnosis.message}</div>
const run = diagnosis.run ?? {}
const steps = Array.isArray(run.steps) ? run.steps : []
const logs = Array.isArray(diagnosis.logs) ? diagnosis.logs : []
return (
<div className="gyxxwb-diagnosis">
<div className="gyxxwb-diagnosis-head">
诊断:{run.run_id}({run.status})
</div>
{steps
.filter((step) => step.status === 'failed')
.map((step) => (
<div key={step.id} className="gyxxwb-step-fail">
<div className="gyxxwb-step-id">
✗ {step.id}(退出码 {step.exit_code ?? '—'})
</div>
{step.error && <pre className="gyxxwb-pre">{step.error}</pre>}
</div>
))}
{logs.map((log) => (
<div key={log.path} className="gyxxwb-log">
<div className="gyxxwb-log-path">{log.path}</div>
<pre className="gyxxwb-pre">{log.tail}</pre>
</div>
))}
{!steps.some((s) => s.status === 'failed') && !logs.length && (
<div className="gyxxwb-empty">该运行没有失败步骤或日志</div>
)}
</div>
)
}
export function WorkflowPanel() {
const s = useWorkbench()
useEffect(() => {
if (!s.open) return undefined
refreshWorkflows()
const timer = setInterval(refreshWorkflows, 10_000)
return () => clearInterval(timer)
}, [s.open])
const onSelect = useCallback((workflowId) => {
selectWorkflow(workflowId)
}, [])
const groups = []
const byModule = new Map()
for (const workflow of s.workflows) {
const key = workflow.module_label ?? workflow.module
if (!byModule.has(key)) byModule.set(key, [])
byModule.get(key).push(workflow)
}
for (const [label, items] of byModule) groups.push({ label, items })
const selected = s.workflows.find((w) => w.id === s.selectedId) ?? null
return (
<div className={`gyxxwb-panel ${s.open ? 'gyxxwb-panel-open' : ''}`}>
<div className="gyxxwb-panel-head">
<span className="gyxxwb-title">GYXX 工作流</span>
<SummaryPills summary={s.summary} workflows={s.workflows} />
<button type="button" className="gyxxwb-icon-btn" title="刷新" onClick={refreshWorkflows}>
⟳
</button>
<button
type="button"
className="gyxxwb-icon-btn"
title="关闭"
onClick={() => update({ open: false })}
>
✕
</button>
</div>
{s.error && (
<div className="gyxxwb-error">
无法连接工作台桥接服务:{s.error}
<br />
请确认 dsh 已通过 workbench/cordis.yml 启动。
</div>
)}
<div className="gyxxwb-panel-body">
<div className="gyxxwb-list">
{groups.map((group) => (
<div key={group.label} className="gyxxwb-group">
<div className="gyxxwb-group-label">{group.label}</div>
{group.items.map((workflow) => (
<WorkflowRow
key={workflow.id}
workflow={workflow}
selected={workflow.id === s.selectedId}
onSelect={onSelect}
/>
))}
</div>
))}
{!s.loading && !s.workflows.length && !s.error && (
<div className="gyxxwb-empty">未发现工作流</div>
)}
</div>
{selected && (
<div className="gyxxwb-detail">
<div className="gyxxwb-detail-title">
{selected.name}
<span className={`gyxxwb-status gyxxwb-status-${statusOf(selected)}`}>
{STATUS_LABELS[statusOf(selected)]}
</span>
</div>
<div className="gyxxwb-detail-sub">
{selected.id} · 调度{' '}
{selected.schedule
? `${selected.schedule.enabled ? '启用' : '停用'} ${
Array.isArray(selected.schedule.at)
? selected.schedule.at.join('/')
: (selected.schedule.at ?? '')
}`
: '无'}
{selected.next_run_at ? ` · 下次 ${formatTime(selected.next_run_at)}` : ''}
</div>
<div className="gyxxwb-actions">
<button type="button" onClick={() => doAsk('diagnose')}>诊断</button>
<button type="button" onClick={() => doAsk('repair')}>修复</button>
<button type="button" onClick={() => doTrigger(false)}>试运行</button>
<button type="button" className="gyxxwb-danger" onClick={() => doTrigger(true)}>
正式运行
</button>
{selected.active_run && (
<button type="button" className="gyxxwb-danger" onClick={doCancel}>
停止
</button>
)}
</div>
<div className="gyxxwb-ask">
<input
value={s.question}
placeholder="就这个工作流提问,例如:昨天为什么失败?"
onChange={(event) => update({ question: event.target.value })}
onKeyDown={(event) => {
if (event.key === 'Enter') doAsk('ask')
}}
/>
<button type="button" onClick={() => doAsk('ask')}>提问</button>
</div>
<div className="gyxxwb-date-row">
业务日期{' '}
<input
value={s.businessDate}
onChange={(event) => update({ businessDate: event.target.value })}
pattern="\d{4}-\d{2}-\d{2}"
/>
</div>
<div className="gyxxwb-section-label">最近运行</div>
<RunList runs={s.runs} onDiagnose={(runId) => loadDiagnosis(selected.id, runId)} />
<DiagnosisView diagnosis={s.diagnosis} loading={s.diagnosisLoading} />
</div>
)}
</div>
</div>
)
}
-74
View File
@@ -1,74 +0,0 @@
/** 失败告警浮层:轮询桥接告警,新失败弹出 toast,可一键发起诊断会话。 */
import { useEffect } from 'react'
import { useSyncExternalStore } from 'react'
import { getState, subscribe, update, dismissToast, pushToast } from './store.js'
import { api } from './api.js'
function useWorkbench() {
return useSyncExternalStore(subscribe, getState)
}
export function Toasts() {
const s = useWorkbench()
useEffect(() => {
let stopped = false
let timer = null
const poll = async () => {
try {
const payload = await api.alerts()
const fresh = (payload.alerts ?? []).filter(
(alert) => !alert.seen && !getState().seenAlertIds[alert.id],
)
if (fresh.length) {
const seen = { ...getState().seenAlertIds }
for (const alert of fresh) {
seen[alert.id] = true
pushToast({ kind: 'failed', text: `工作流「${alert.name}」运行失败`, alert })
}
update({ seenAlertIds: seen })
}
} catch {
// 桥接离线时静默
}
if (!stopped) timer = setTimeout(poll, 30_000)
}
timer = setTimeout(poll, 8_000)
return () => {
stopped = true
if (timer) clearTimeout(timer)
}
}, [])
if (!s.toasts.length) return null
return (
<div className="gyxxwb-toasts">
{s.toasts.map((toast) => (
<div key={toast.id} className={`gyxxwb-toast gyxxwb-toast-${toast.kind}`}>
<span className="gyxxwb-toast-text">{toast.text}</span>
{toast.alert && (
<button
type="button"
onClick={async () => {
try {
await api.ask({ workflow_id: toast.alert.workflow_id, action: 'diagnose' })
await api.markAlertSeen(toast.alert.id).catch(() => {})
pushToast({ kind: 'ok', text: '已创建诊断会话,请在左侧会话列表查看' })
} catch (error) {
pushToast({ kind: 'error', text: `创建会话失败:${error.message}` })
}
dismissToast(toast.id)
}}
>
立即诊断
</button>
)}
<button type="button" title="关闭" onClick={() => dismissToast(toast.id)}>
✕
</button>
</div>
))}
</div>
)
}
-55
View File
@@ -1,55 +0,0 @@
/** 工作台桥接服务客户端:所有数据经由 gyxx-workbench 宿主插件的本机回环桥接。 */
export function bridgeBase() {
if (typeof window !== 'undefined') {
if (window.__GYXX_WB_BRIDGE__) return window.__GYXX_WB_BRIDGE__
try {
const saved = window.localStorage?.getItem('gyxx.wb.bridge')
if (saved) return saved
} catch {
// localStorage 不可用时使用默认值
}
}
return 'http://127.0.0.1:8790'
}
async function request(method, path, body) {
const response = await fetch(`${bridgeBase()}${path}`, {
method,
headers: {
accept: 'application/json',
...(body !== undefined
? { 'content-type': 'application/json', 'x-gyxx-workbench': '1' }
: {}),
},
body: body === undefined ? undefined : JSON.stringify(body),
})
let payload
try {
payload = await response.json()
} catch {
payload = { ok: false, error: `桥接服务返回了非 JSON 内容(HTTP ${response.status})` }
}
if (!response.ok) {
throw new Error(payload?.error ?? `桥接服务错误(HTTP ${response.status})`)
}
return payload
}
export const api = {
listWorkflows: () => request('GET', '/bridge/workflows'),
workflowDetail: (workflowId) =>
request('GET', `/bridge/workflows/${encodeURIComponent(workflowId)}`),
workflowRuns: (workflowId, limit = 6) =>
request('GET', `/bridge/workflows/${encodeURIComponent(workflowId)}/runs?limit=${limit}`),
diagnosis: (workflowId, runId) =>
request(
'GET',
`/bridge/workflows/${encodeURIComponent(workflowId)}/diagnosis${runId ? `?run_id=${encodeURIComponent(runId)}` : ''}`,
),
alerts: () => request('GET', '/bridge/alerts'),
markAlertSeen: (id) => request('POST', '/bridge/alerts/seen', { id }),
trigger: (payload) => request('POST', '/bridge/trigger', payload),
cancel: (payload) => request('POST', '/bridge/cancel', payload),
ask: (payload) => request('POST', '/bridge/ask', payload),
}
-85
View File
@@ -1,85 +0,0 @@
/**
* GYXX 工作台客户端插件入口。
* 由 esbuild 打包为 CJS 并通过 window.__ModuleLoader__ 注册(见 scripts/build-client.mjs)。
* 运行时只依赖 dsh 平台共享模块表中的 react。
*/
import { createElement, useSyncExternalStore } from 'react'
import { WorkflowPanel, refreshWorkflows } from './Panel.jsx'
import { Toasts } from './Toasts.jsx'
import { getState, subscribe, update } from './store.js'
import stylesText from './styles.css'
export const inject = ['slots']
function useWorkbench() {
return useSyncExternalStore(subscribe, getState)
}
function injectStyles() {
if (typeof document === 'undefined') return
if (document.getElementById('gyxxwb-styles')) return
const style = document.createElement('style')
style.id = 'gyxxwb-styles'
style.textContent = stylesText
document.head.appendChild(style)
}
/** 侧边栏底部动作:工作流面板开关 + 聚合状态点。 */
function WorkbenchFooterAction() {
const s = useWorkbench()
const failed = s.workflows.filter((w) => !w.active_run && w.last_run?.status === 'failed').length
const running = s.workflows.filter((w) => w.active_run).length
const tone = failed > 0 ? 'failed' : running > 0 ? 'running' : 'success'
return createElement(
'button',
{
type: 'button',
className: `gyxxwb-footer-btn${s.open ? ' gyxxwb-footer-btn-active' : ''}`,
title: 'GYXX 工作流工作台',
onClick: () => {
const next = !getState().open
update({ open: next })
if (next) refreshWorkflows()
},
},
createElement('span', { className: `gyxxwb-dot gyxxwb-dot-${tone}` }),
createElement('span', null, '工作流'),
)
}
/** 覆盖层根:左侧抽屉面板 + 失败 toast。 */
function WorkbenchOverlay() {
return createElement(
'div',
{ className: 'gyxxwb-overlay' },
createElement(WorkflowPanel, null),
createElement(Toasts, null),
)
}
export function apply(ctx) {
injectStyles()
// 插槽须等宿主声明后才能注册:slots.inject 会等待座位声明(顺序安全),
// 直接 register 在我们先于 ui-sidebar/ui-layout 加载时会报“slot is not declared”。
ctx.effect(
() =>
ctx.slots.inject('sidebar.footer.action', () =>
ctx.slots.register(
{ name: 'sidebar.footer.action', id: 'gyxx-workbench.footer' },
WorkbenchFooterAction,
),
),
'gyxx-workbench: footer action',
)
ctx.effect(
() =>
ctx.slots.inject('shell.overlay', () =>
ctx.slots.register(
{ name: 'shell.overlay', id: 'gyxx-workbench.overlay' },
WorkbenchOverlay,
),
),
'gyxx-workbench: overlay',
)
}
-57
View File
@@ -1,57 +0,0 @@
/**
* GYXX 工作台面板 —— 极简外部存储(避免向客户端包引入额外依赖)。
* React 18 的 useSyncExternalStore 直接订阅。
*/
const listeners = new Set()
let state = {
open: false,
loading: false,
error: null,
generatedAt: null,
summary: null,
workflows: [],
selectedId: null,
detail: null,
detailLoading: false,
runs: [],
diagnosis: null,
diagnosisLoading: false,
question: '',
businessDate: defaultBusinessDate(),
alerts: [],
seenAlertIds: {},
toasts: [],
}
export function defaultBusinessDate() {
const now = new Date()
now.setDate(now.getDate() - 1)
const pad = (value) => String(value).padStart(2, '0')
return `${now.getFullYear()}-${pad(now.getMonth() + 1)}-${pad(now.getDate())}`
}
export function getState() {
return state
}
export function update(patch) {
state = { ...state, ...patch }
for (const listener of listeners) listener()
}
export function subscribe(listener) {
listeners.add(listener)
return () => listeners.delete(listener)
}
export function pushToast(toast) {
const id = `t-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 6)}`
update({ toasts: [...state.toasts, { id, ...toast }] })
return id
}
export function dismissToast(id) {
update({ toasts: state.toasts.filter((toast) => toast.id !== id) })
}
-471
View File
@@ -1,471 +0,0 @@
/* GYXX 工作台面板样式 —— 以文本形式内联进客户端包,运行时注入 <style>。 */
.gyxxwb-overlay {
position: fixed;
inset: 0;
pointer-events: none;
z-index: 70;
font-family:
-apple-system, BlinkMacSystemFont, 'Segoe UI', 'PingFang SC',
'Hiragino Sans GB', 'Microsoft YaHei', sans-serif;
}
.gyxxwb-panel {
position: absolute;
top: 0;
bottom: 0;
left: 264px;
width: 380px;
display: none;
flex-direction: column;
background: #ffffff;
border-right: 1px solid #e4e4e7;
box-shadow: 8px 0 24px rgba(0, 0, 0, 0.08);
pointer-events: auto;
color: #18181b;
}
.gyxxwb-panel-open {
display: flex;
}
.gyxxwb-panel-head {
display: flex;
align-items: center;
gap: 8px;
padding: 10px 12px;
border-bottom: 1px solid #e4e4e7;
}
.gyxxwb-title {
font-weight: 600;
font-size: 14px;
}
.gyxxwb-pills {
display: flex;
gap: 4px;
flex: 1;
}
.gyxxwb-pill {
font-size: 11px;
padding: 1px 8px;
border-radius: 999px;
background: #f4f4f5;
color: #52525b;
}
.gyxxwb-pill-failed {
background: #fef2f2;
color: #b91c1c;
}
.gyxxwb-pill-running {
background: #eff6ff;
color: #1d4ed8;
}
.gyxxwb-icon-btn {
border: none;
background: transparent;
cursor: pointer;
font-size: 14px;
color: #71717a;
padding: 4px 6px;
border-radius: 6px;
}
.gyxxwb-icon-btn:hover {
background: #f4f4f5;
}
.gyxxwb-error {
margin: 10px 12px;
padding: 10px;
border-radius: 8px;
background: #fef2f2;
color: #b91c1c;
font-size: 12px;
line-height: 1.6;
}
.gyxxwb-panel-body {
flex: 1;
overflow-y: auto;
display: flex;
flex-direction: column;
}
.gyxxwb-group-label {
padding: 10px 12px 4px;
font-size: 12px;
color: #71717a;
}
.gyxxwb-row {
display: flex;
align-items: center;
gap: 8px;
width: 100%;
padding: 7px 12px;
border: none;
background: transparent;
cursor: pointer;
text-align: left;
}
.gyxxwb-row:hover {
background: #f4f4f5;
}
.gyxxwb-row-selected {
background: #eef2ff;
}
.gyxxwb-row-main {
flex: 1;
min-width: 0;
display: flex;
flex-direction: column;
}
.gyxxwb-row-name {
font-size: 13px;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.gyxxwb-row-id {
font-size: 11px;
color: #a1a1aa;
}
.gyxxwb-row-meta {
font-size: 11px;
color: #71717a;
}
.gyxxwb-dot {
width: 8px;
height: 8px;
border-radius: 50%;
background: #d4d4d8;
flex: none;
}
.gyxxwb-dot-success {
background: #16a34a;
}
.gyxxwb-dot-failed {
background: #dc2626;
}
.gyxxwb-dot-running {
background: #2563eb;
animation: gyxxwb-pulse 1.2s ease-in-out infinite;
}
.gyxxwb-dot-cancelled,
.gyxxwb-dot-none {
background: #a1a1aa;
}
@keyframes gyxxwb-pulse {
50% {
opacity: 0.35;
}
}
.gyxxwb-detail {
border-top: 1px solid #e4e4e7;
padding: 12px;
background: #fafafa;
}
.gyxxwb-detail-title {
font-size: 14px;
font-weight: 600;
display: flex;
align-items: center;
gap: 8px;
}
.gyxxwb-status {
font-size: 11px;
padding: 1px 8px;
border-radius: 999px;
background: #f4f4f5;
}
.gyxxwb-status-failed {
background: #fef2f2;
color: #b91c1c;
}
.gyxxwb-status-running {
background: #eff6ff;
color: #1d4ed8;
}
.gyxxwb-status-success {
background: #f0fdf4;
color: #15803d;
}
.gyxxwb-detail-sub {
font-size: 11px;
color: #71717a;
margin: 4px 0 10px;
}
.gyxxwb-actions {
display: flex;
flex-wrap: wrap;
gap: 6px;
}
.gyxxwb-actions button,
.gyxxwb-ask button {
border: 1px solid #e4e4e7;
background: #ffffff;
border-radius: 6px;
font-size: 12px;
padding: 4px 10px;
cursor: pointer;
}
.gyxxwb-actions button:hover,
.gyxxwb-ask button:hover {
background: #f4f4f5;
}
.gyxxwb-actions .gyxxwb-danger {
border-color: #fecaca;
color: #b91c1c;
}
.gyxxwb-ask {
display: flex;
gap: 6px;
margin-top: 8px;
}
.gyxxwb-ask input {
flex: 1;
border: 1px solid #e4e4e7;
border-radius: 6px;
padding: 5px 8px;
font-size: 12px;
}
.gyxxwb-date-row {
margin-top: 8px;
font-size: 11px;
color: #71717a;
display: flex;
align-items: center;
gap: 6px;
}
.gyxxwb-date-row input {
border: 1px solid #e4e4e7;
border-radius: 6px;
padding: 3px 6px;
font-size: 11px;
width: 110px;
}
.gyxxwb-section-label {
margin: 12px 0 4px;
font-size: 12px;
color: #71717a;
}
.gyxxwb-run {
display: flex;
align-items: center;
gap: 8px;
width: 100%;
padding: 6px 8px;
border: none;
background: transparent;
cursor: pointer;
font-size: 12px;
text-align: left;
border-radius: 6px;
}
.gyxxwb-run:hover {
background: #f4f4f5;
}
.gyxxwb-run-mode {
color: #a1a1aa;
font-size: 11px;
}
.gyxxwb-run-error {
color: #b91c1c;
font-size: 11px;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.gyxxwb-diagnosis {
margin-top: 10px;
}
.gyxxwb-diagnosis-head {
font-size: 12px;
font-weight: 600;
margin-bottom: 6px;
}
.gyxxwb-step-fail {
margin-bottom: 8px;
}
.gyxxwb-step-id {
font-size: 12px;
color: #b91c1c;
margin-bottom: 2px;
}
.gyxxwb-log {
margin-bottom: 8px;
}
.gyxxwb-log-path {
font-size: 11px;
color: #71717a;
margin-bottom: 2px;
word-break: break-all;
}
.gyxxwb-pre {
font-size: 11px;
background: #18181b;
color: #e4e4e7;
border-radius: 6px;
padding: 8px;
max-height: 180px;
overflow: auto;
white-space: pre-wrap;
word-break: break-all;
}
.gyxxwb-empty {
padding: 16px 12px;
font-size: 12px;
color: #a1a1aa;
text-align: center;
}
.gyxxwb-toasts {
position: absolute;
right: 16px;
bottom: 16px;
display: flex;
flex-direction: column;
gap: 8px;
pointer-events: auto;
max-width: 360px;
}
.gyxxwb-toast {
display: flex;
align-items: center;
gap: 8px;
background: #18181b;
color: #fafafa;
border-radius: 10px;
padding: 10px 12px;
font-size: 12px;
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.2);
}
.gyxxwb-toast button {
border: 1px solid #3f3f46;
background: transparent;
color: #fafafa;
border-radius: 6px;
font-size: 11px;
padding: 2px 8px;
cursor: pointer;
}
.gyxxwb-toast-ok {
background: #14532d;
}
.gyxxwb-toast-error,
.gyxxwb-toast-failed {
background: #7f1d1d;
}
.gyxxwb-footer-btn {
display: inline-flex;
align-items: center;
gap: 6px;
border: none;
background: transparent;
cursor: pointer;
font-size: 13px;
color: inherit;
padding: 6px 8px;
border-radius: 8px;
}
.gyxxwb-footer-btn:hover,
.gyxxwb-footer-btn-active {
background: rgba(0, 0, 0, 0.06);
}
@media (max-width: 900px) {
.gyxxwb-panel {
left: 0;
width: 100%;
}
}
@media (prefers-color-scheme: dark) {
.gyxxwb-panel {
background: #18181b;
color: #fafafa;
border-right-color: #3f3f46;
}
.gyxxwb-row:hover {
background: #27272a;
}
.gyxxwb-row-selected {
background: #312e81;
}
.gyxxwb-detail {
background: #1f1f22;
border-top-color: #3f3f46;
}
.gyxxwb-actions button,
.gyxxwb-ask button {
background: #27272a;
border-color: #3f3f46;
color: #fafafa;
}
.gyxxwb-ask input,
.gyxxwb-date-row input {
background: #27272a;
border-color: #3f3f46;
color: #fafafa;
}
.gyxxwb-pill {
background: #27272a;
color: #a1a1aa;
}
.gyxxwb-footer-btn:hover,
.gyxxwb-footer-btn-active {
background: rgba(255, 255, 255, 0.08);
}
}
-258
View File
@@ -1,258 +0,0 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>GYXX 智能工作台</title>
<style>
:root { color-scheme: light dark; }
* { box-sizing: border-box; }
body {
margin: 0; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'PingFang SC',
'Hiragino Sans GB', 'Microsoft YaHei', sans-serif;
background: #f4f4f5; color: #18181b;
}
header {
display: flex; align-items: center; gap: 10px; padding: 12px 16px;
background: #fff; border-bottom: 1px solid #e4e4e7; position: sticky; top: 0; z-index: 2;
}
h1 { font-size: 15px; margin: 0; }
.pill { font-size: 11px; padding: 1px 8px; border-radius: 999px; background: #f4f4f5; color: #52525b; border: 1px solid #e4e4e7; }
.pill-failed { background: #fef2f2; color: #b91c1c; border-color: #fecaca; }
.pill-running { background: #eff6ff; color: #1d4ed8; border-color: #bfdbfe; }
main { display: flex; gap: 0; min-height: calc(100vh - 49px); }
#list { width: 320px; background: #fff; border-right: 1px solid #e4e4e7; overflow-y: auto; }
#detail { flex: 1; padding: 16px; overflow-y: auto; }
.group-label { padding: 10px 12px 4px; font-size: 12px; color: #71717a; }
.row { display: flex; align-items: center; gap: 8px; width: 100%; padding: 8px 12px; border: none; background: none; cursor: pointer; text-align: left; }
.row:hover { background: #f4f4f5; }
.row.selected { background: #eef2ff; }
.row .name { font-size: 13px; display: block; }
.row .id { font-size: 11px; color: #a1a1aa; display: block; }
.row .meta { font-size: 11px; color: #71717a; margin-left: auto; }
.dot { width: 8px; height: 8px; border-radius: 50%; background: #a1a1aa; flex: none; }
.dot-success { background: #16a34a; } .dot-failed { background: #dc2626; }
.dot-running { background: #2563eb; animation: pulse 1.2s infinite; }
@keyframes pulse { 50% { opacity: .35; } }
button.act { border: 1px solid #e4e4e7; background: #fff; border-radius: 6px; font-size: 12px; padding: 5px 12px; cursor: pointer; margin-right: 6px; }
button.act:hover { background: #f4f4f5; }
button.danger { border-color: #fecaca; color: #b91c1c; }
input { border: 1px solid #e4e4e7; border-radius: 6px; padding: 6px 8px; font-size: 12px; }
pre { background: #18181b; color: #e4e4e7; border-radius: 8px; padding: 10px; font-size: 11px; max-height: 260px; overflow: auto; white-space: pre-wrap; word-break: break-all; }
.muted { color: #71717a; font-size: 12px; }
.err { color: #b91c1c; font-size: 12px; }
.card { background: #fff; border: 1px solid #e4e4e7; border-radius: 10px; padding: 14px; margin-bottom: 12px; }
#toasts { position: fixed; right: 16px; bottom: 16px; display: flex; flex-direction: column; gap: 8px; z-index: 9; }
.toast { background: #18181b; color: #fafafa; border-radius: 10px; padding: 10px 12px; font-size: 12px; max-width: 340px; }
.toast.ok { background: #14532d; } .toast.err { background: #7f1d1d; }
@media (prefers-color-scheme: dark) {
body { background: #09090b; color: #fafafa; }
header, #list, .card { background: #18181b; border-color: #3f3f46; }
.row:hover { background: #27272a; } .row.selected { background: #312e81; }
button.act, input { background: #27272a; border-color: #3f3f46; color: #fafafa; }
}
</style>
</head>
<body>
<header>
<h1>GYXX 智能工作台</h1>
<span id="pills"></span>
<span class="muted" style="margin-left:auto" id="generated"></span>
<button class="act" onclick="loadWorkflows()">刷新</button>
</header>
<main>
<div id="list"></div>
<div id="detail"><p class="muted" style="padding:16px">在左侧选择一个工作流,可提问、诊断、修复或启动。</p></div>
</main>
<div id="toasts"></div>
<script>
const $ = (sel) => document.querySelector(sel)
const state = { workflows: [], selected: null, detail: null, runs: [], seenAlerts: {} }
function toast(text, kind) {
const el = document.createElement('div')
el.className = 'toast ' + (kind || '')
el.textContent = text
$('#toasts').appendChild(el)
setTimeout(() => el.remove(), 8000)
}
async function req(method, path, body) {
const res = await fetch(path, {
method,
headers: body !== undefined ? { 'content-type': 'application/json', 'x-gyxx-workbench': '1' } : {},
body: body === undefined ? undefined : JSON.stringify(body),
})
const payload = await res.json().catch(() => ({}))
if (!res.ok) throw new Error(payload.error || ('HTTP ' + res.status))
return payload
}
function statusOf(w) { return w.active_run ? 'running' : (w.last_run && w.last_run.status) || 'none' }
function fmtTime(v) {
if (!v) return '—'
const d = new Date(v); if (isNaN(d)) return v
const p = (n) => String(n).padStart(2, '0')
return `${p(d.getMonth()+1)}-${p(d.getDate())} ${p(d.getHours())}:${p(d.getMinutes())}`
}
function esc(s) { return String(s == null ? '' : s).replace(/[&<>"]/g, (c) => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;'}[c])) }
async function loadWorkflows() {
try {
const payload = await req('GET', '/bridge/workflows')
state.workflows = payload.workflows || []
$('#generated').textContent = payload.generated_at ? ('更新于 ' + fmtTime(payload.generated_at)) : ''
renderList()
} catch (e) {
$('#list').innerHTML = '<p class="err" style="padding:12px">无法连接桥接服务:' + esc(e.message) + '<br>请确认 gyxx console 与 dsh workbench 插件已启动。</p>'
}
}
function renderList() {
const failed = state.workflows.filter((w) => statusOf(w) === 'failed').length
const running = state.workflows.filter((w) => w.active_run).length
$('#pills').innerHTML =
`<span class="pill">共 ${state.workflows.length}</span>` +
(running ? `<span class="pill pill-running">运行中 ${running}</span>` : '') +
(failed ? `<span class="pill pill-failed">失败 ${failed}</span>` : '')
const byModule = new Map()
for (const w of state.workflows) {
const key = w.module_label || w.module
if (!byModule.has(key)) byModule.set(key, [])
byModule.get(key).push(w)
}
let html = ''
for (const [label, items] of byModule) {
html += `<div class="group-label">${esc(label)}</div>`
for (const w of items) {
const st = statusOf(w)
html += `<button class="row ${state.selected === w.id ? 'selected' : ''}" onclick="select('${esc(w.id)}')">
<span class="dot dot-${st}"></span>
<span><span class="name">${esc(w.name)}</span><span class="id">${esc(w.id)}</span></span>
<span class="meta">${w.last_run ? fmtTime(w.last_run.ended_at || w.last_run.started_at) : '—'}</span>
</button>`
}
}
$('#list').innerHTML = html || '<p class="muted" style="padding:12px">未发现工作流</p>'
}
async function select(id) {
state.selected = id
renderList()
$('#detail').innerHTML = '<p class="muted" style="padding:16px">加载中…</p>'
try {
const [detail, runs] = await Promise.all([
req('GET', '/bridge/workflows/' + encodeURIComponent(id)).catch(() => null),
req('GET', '/bridge/workflows/' + encodeURIComponent(id) + '/runs?limit=6').catch(() => ({ runs: [] })),
])
state.detail = detail
state.runs = runs.runs || []
renderDetail()
} catch (e) {
$('#detail').innerHTML = '<p class="err" style="padding:16px">' + esc(e.message) + '</p>'
}
}
function renderDetail() {
const w = state.workflows.find((x) => x.id === state.selected)
if (!w) return
const yesterday = new Date(); yesterday.setDate(yesterday.getDate() - 1)
const p = (n) => String(n).padStart(2, '0')
const defaultDate = `${yesterday.getFullYear()}-${p(yesterday.getMonth()+1)}-${p(yesterday.getDate())}`
const sched = w.schedule
let html = `<div class="card">
<h2 style="font-size:15px;margin:0 0 4px">${esc(w.name)}</h2>
<div class="muted">${esc(w.id)} · 调度 ${sched ? (sched.enabled ? '启用' : '停用') + ' ' + (Array.isArray(sched.at) ? sched.at.join('/') : sched.at || '') : '无'}</div>
<div style="margin:10px 0">
<button class="act" onclick="ask('diagnose')">诊断</button>
<button class="act" onclick="ask('repair')">修复</button>
<button class="act" onclick="triggerRun(false)">试运行</button>
<button class="act danger" onclick="triggerRun(true)">正式运行</button>
${w.active_run ? '<button class="act danger" onclick="cancelRun()">停止</button>' : ''}
</div>
<div style="display:flex;gap:6px;margin-bottom:8px">
<input id="q" style="flex:1" placeholder="就这个工作流提问,例如:昨天为什么失败?" />
<button class="act" onclick="ask('ask')">提问</button>
</div>
<div class="muted">业务日期 <input id="bd" value="${defaultDate}" style="width:110px" /></div>
</div>
<div class="card"><div class="muted" style="margin-bottom:6px">最近运行(点击查看诊断)</div>`
if (!state.runs.length) html += '<div class="muted">暂无运行记录</div>'
for (const r of state.runs) {
html += `<button class="row" onclick="diagnose('${esc(r.run_id)}')">
<span class="dot dot-${r.status}"></span><span>${esc(r.business_date)}</span>
<span class="muted">${r.mode === 'execute' ? '正式' : '试运行'}</span>
${r.error ? `<span class="err">${esc(r.error.slice(0, 80))}</span>` : ''}
</button>`
}
html += '</div><div id="diag"></div>'
$('#detail').innerHTML = html
}
async function diagnose(runId) {
$('#diag').innerHTML = '<div class="card muted">加载诊断数据…</div>'
try {
const d = await req('GET', '/bridge/workflows/' + encodeURIComponent(state.selected) + '/diagnosis?run_id=' + encodeURIComponent(runId))
let html = '<div class="card"><div class="muted">诊断:' + esc(runId) + '</div>'
const steps = (d.run && d.run.steps) || []
for (const s of steps.filter((x) => x.status === 'failed')) {
html += `<div class="err">✗ ${esc(s.id)}(退出码 ${s.exit_code == null ? '—' : s.exit_code})</div>${s.error ? '<pre>' + esc(s.error) + '</pre>' : ''}`
}
for (const log of d.logs || []) {
html += `<div class="muted" style="margin-top:8px">${esc(log.path)}</div><pre>${esc(log.tail)}</pre>`
}
if (!steps.some((x) => x.status === 'failed') && !(d.logs || []).length) html += '<div class="muted">该运行没有失败步骤或日志</div>'
$('#diag').innerHTML = html + '</div>'
} catch (e) {
$('#diag').innerHTML = '<div class="card err">' + esc(e.message) + '</div>'
}
}
async function ask(action) {
const question = ($('#q') && $('#q').value) || ''
try {
const payload = await req('POST', '/bridge/ask', { workflow_id: state.selected, action, question })
toast('已创建会话(' + String(payload.session_id).slice(0, 24) + '…),请在 dsh 侧边栏会话列表中查看', 'ok')
} catch (e) { toast('创建会话失败:' + e.message, 'err') }
}
async function triggerRun(execute) {
const businessDate = ($('#bd') && $('#bd').value) || ''
if (execute && !confirm(`正式执行 ${state.selected}(业务日期 ${businessDate})会产生真实外部写入。确认继续?`)) return
try {
await req('POST', '/bridge/trigger', { workflow_id: state.selected, business_date: businessDate, execute, confirmed: execute })
toast(execute ? '已发起正式执行' : '已发起试运行(无外部副作用)', 'ok')
setTimeout(loadWorkflows, 1500)
} catch (e) { toast('启动失败:' + e.message, 'err') }
}
async function cancelRun() {
const w = state.workflows.find((x) => x.id === state.selected)
const body = w && w.active_run && w.active_run.operation_id
? { workflow_id: state.selected, operation_id: w.active_run.operation_id }
: { workflow_id: state.selected, scheduled: true }
try {
await req('POST', '/bridge/cancel', body)
toast('已发送停止指令', 'ok')
setTimeout(loadWorkflows, 1500)
} catch (e) { toast('停止失败:' + e.message, 'err') }
}
async function pollAlerts() {
try {
const payload = await req('GET', '/bridge/alerts')
for (const alert of payload.alerts || []) {
if (alert.seen || state.seenAlerts[alert.id]) continue
state.seenAlerts[alert.id] = true
toast('工作流「' + alert.name + '」运行失败,可在 dsh 中发起诊断', 'err')
req('POST', '/bridge/alerts/seen', { id: alert.id }).catch(() => {})
}
} catch (e) { /* 桥接离线时静默 */ }
}
loadWorkflows()
setInterval(loadWorkflows, 15000)
setInterval(pollAlerts, 30000)
</script>
</body>
</html>
-937
View File
@@ -1,937 +0,0 @@
/**
* GYXX 智能工作台 — deepseek-harness 宿主插件。
*
* 通过 gyxx-flow 控制台 HTTP API(默认 http://127.0.0.1:8765)把工作流
* 暴露为 DeepSeek 智能体工具,并提供:
* - 7 个模型可调用的工作流工具(查询 / 诊断 / 触发 / 停止 / 定时配置)
* - 中文系统提示词(安全契约:默认试运行,正式执行必须用户确认)
* - 失败监控器(轮询 overview,发现新失败时发出告警,可选自动开诊断会话)
* - 本机回环桥接服务(默认 127.0.0.1:8790),供侧边栏面板读取数据、
* 发起「提问 / 诊断 / 修复」会话;同时内置一个零依赖的独立面板页面
* (GET /)作为 dsh 客户端插件加载失败时的降级 UI。
*
* 本文件零运行时依赖(只用 Node 内置模块),经 cordis.yml --patch 以绝对
* 路径插入 dsh Web 组合即可运行。
*/
import http from 'node:http'
import { randomUUID } from 'node:crypto'
import { readFileSync } from 'node:fs'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
export const name = 'gyxx-workbench'
export const inject = ['tools', 'systemPrompt']
const PLUGIN_DIR = dirname(fileURLToPath(import.meta.url))
const MODULE_LABELS = {
content_marketing: '内容营销',
product_commerce: '商品经营',
shop_intelligence: '店铺情报',
supply_chain: '供应链',
}
/* ------------------------------------------------------------------ */
/* 控制台 HTTP 客户端 */
/* ------------------------------------------------------------------ */
class ConsoleApiError extends Error {
constructor(status, message) {
super(message)
this.name = 'ConsoleApiError'
this.status = status
}
}
function makeConsoleClient(config) {
const base = new URL(config.consoleBaseUrl)
return async function consoleApi(method, path, body, options = {}) {
const headers = { accept: 'application/json' }
if (config.consoleToken) headers.authorization = `Bearer ${config.consoleToken}`
if (body !== undefined) {
headers['content-type'] = 'application/json'
headers['x-gyxx-console'] = '1'
}
if (options.ifMatch) headers['if-match'] = options.ifMatch
let response
try {
response = await fetch(new URL(path, base), {
method,
headers,
body: body === undefined ? undefined : JSON.stringify(body),
signal: options.signal,
})
} catch (error) {
if (error?.name === 'AbortError') throw error
throw new ConsoleApiError(
0,
`无法连接 gyxx 控制台 ${base.origin},请先启动 gyxx console(${error?.message ?? error})`,
)
}
const text = await response.text()
let payload
try {
payload = JSON.parse(text)
} catch {
payload = { raw: text }
}
if (!response.ok) {
const message =
payload && typeof payload.error === 'string'
? payload.error
: `控制台返回 HTTP ${response.status}`
throw new ConsoleApiError(response.status, message)
}
return payload
}
}
/* ------------------------------------------------------------------ */
/* 数据压缩:把控制台载荷裁剪为适合模型阅读的体积 */
/* ------------------------------------------------------------------ */
function compactRun(run) {
if (!run) return null
const compact = {
run_id: run.run_id,
business_date: run.business_date,
mode: run.mode,
shadow: run.shadow,
status: run.status,
started_at: run.started_at,
ended_at: run.ended_at,
duration_seconds: run.duration_seconds,
error: run.error,
}
if (Array.isArray(run.steps)) {
compact.steps = run.steps.map((step) => ({
id: step.id,
status: step.status,
exit_code: step.exit_code,
error: step.error,
}))
}
return compact
}
function compactWorkflow(item) {
const schedule = item.schedule ?? null
return {
id: item.id,
name: item.name,
module: item.module,
module_label: MODULE_LABELS[item.module] ?? item.module,
trigger: item.trigger,
registered: item.registered,
note: item.note ?? null,
schedule: schedule
? {
enabled: schedule.enabled,
kind: schedule.kind,
at: schedule.at,
raw: schedule,
}
: null,
next_run_at: item.next_run_at ?? null,
last_run: compactRun(item.last_run),
active_run: item.active_run
? {
operation_id: item.active_run.operation_id,
business_date: item.active_run.business_date,
mode: item.active_run.mode,
started_at: item.active_run.started_at,
}
: null,
}
}
function compactDetail(payload) {
const workflow = payload.workflow ?? {}
const steps = Array.isArray(workflow.steps)
? workflow.steps.map((step) => ({
id: step.id,
name: step.name,
description: step.description,
entry: step.entry,
timeout_seconds: step.timeout_seconds,
replay_policy: step.replay_policy,
depends_on: step.depends_on,
}))
: []
return {
workflow: { ...compactWorkflow(workflow), steps },
schedule_revision: payload.schedule_revision ?? null,
warnings: payload.warnings ?? [],
}
}
/* ------------------------------------------------------------------ */
/* 系统提示词 */
/* ------------------------------------------------------------------ */
const SYSTEM_PROMPT_TEXT = `\
你是 GYXX 智能工作台的运维智能体。gyxx-flow 是一套 LangGraph 工作流编排系统,
包含四个业务模块:内容营销(content_marketing)、商品经营(product_commerce)、
店铺情报(shop_intelligence)、供应链(supply_chain)。每个工作流由若干步骤组成,
运行日志(RunJournal)与运行索引位于数据根目录下。
你可以使用以下工具(数据来自本机 gyxx 控制台 HTTP API):
- gyxx_workflow_list:列出全部工作流及其调度、最近一次运行状态。
- gyxx_workflow_detail:查看单个工作流的定义、步骤与调度详情。
- gyxx_workflow_runs:查看某工作流的最近运行历史(含每个步骤的退出码与错误)。
- gyxx_workflow_diagnose:获取一次运行的完整诊断包(日志尾部已脱敏)。
- gyxx_workflow_trigger:触发一次运行。默认 execute=false 为试运行(无副作用)。
- gyxx_workflow_cancel:停止正在运行的手动执行或定时执行。
- gyxx_schedule_update:启用/停用或修改定时调度(读-改-写,需要 schedule_revision)。
安全契约(必须遵守):
1. 默认只做只读分析与试运行。execute=true 的正式执行会对外部系统
(PostgreSQL、飞书、电商平台)产生真实写入,只有在用户于对话中明确确认后,
才能把 confirmed=true 一并传入;否则工具会拒绝。
2. 修复工作流时遵循「诊断 → 定位根因 → 给出方案 → 用户确认 → 执行修复」的顺序,
不得跳过确认直接修改调度或正式重跑。
3. 常见修复手段:登录态失效(提示用户运行 gyxx accounts login/sync)、
幂等冲突(force_refresh 需真实执行且谨慎)、参数错误(改动态配置或调度)、
临时性失败(先试运行验证再正式重跑)。
4. 诊断结果用中文输出,包含:失败步骤、根因推测、建议操作、风险说明。
当用户从工作台侧边栏选中工作流发起提问/诊断/修复时,会话的第一条消息会
带有工作流上下文,请直接开始分析,不要反问用户基本信息。`
/* ------------------------------------------------------------------ */
/* 工具定义 */
/* ------------------------------------------------------------------ */
const JSON_OUTPUT = {
schema: { type: 'object' },
render: (_args, value) => [
{ type: 'text', text: typeof value === 'string' ? value : JSON.stringify(value, null, 2) },
],
}
function toolErrorResult(error) {
return {
ok: false,
error: error instanceof ConsoleApiError ? error.message : `工具执行失败:${error?.message ?? error}`,
}
}
function defineWorkflowTools(ctx, consoleApi) {
const register = (definition) => ctx.tools.register(definition)
register({
name: 'gyxx_workflow_list',
description:
'列出 GYXX 全部调度工作流:名称、模块、调度时间、最近一次运行状态与进行中的执行。',
parameters: {
type: 'object',
properties: {
module: {
type: 'string',
enum: Object.keys(MODULE_LABELS),
description: '按业务模块过滤,缺省返回全部',
},
status: {
type: 'string',
enum: ['failed', 'running', 'success', 'none'],
description: '按最近一次运行状态过滤(none = 从未运行)',
},
},
additionalProperties: false,
},
output: JSON_OUTPUT,
async execute(args, exec) {
try {
const overview = await consoleApi('GET', '/api/overview', undefined, {
signal: exec?.signal,
})
let workflows = (overview.workflows ?? []).map(compactWorkflow)
if (args?.module) workflows = workflows.filter((w) => w.module === args.module)
if (args?.status) {
workflows = workflows.filter((w) => {
const status = w.active_run ? 'running' : (w.last_run?.status ?? 'none')
return args.status === 'running' ? status === 'running' || w.active_run : status === args.status
})
}
return { ok: true, summary: overview.summary ?? null, workflows }
} catch (error) {
return toolErrorResult(error)
}
},
})
register({
name: 'gyxx_workflow_detail',
description: '查看单个工作流的完整定义:步骤、调度、依赖、重放策略与最近运行。',
parameters: {
type: 'object',
properties: {
workflow_id: { type: 'string', description: '工作流 ID,例如 content.metrics.daily' },
},
required: ['workflow_id'],
additionalProperties: false,
},
output: JSON_OUTPUT,
async execute(args, exec) {
try {
const payload = await consoleApi(
'GET',
`/api/workflows/${encodeURIComponent(args.workflow_id)}`,
undefined,
{ signal: exec?.signal },
)
return { ok: true, ...compactDetail(payload) }
} catch (error) {
return toolErrorResult(error)
}
},
})
register({
name: 'gyxx_workflow_runs',
description: '查看某工作流最近的运行历史(含步骤级状态、退出码与脱敏错误)。',
parameters: {
type: 'object',
properties: {
workflow_id: { type: 'string' },
limit: { type: 'integer', minimum: 1, maximum: 20, description: '默认 8,最大 20' },
},
required: ['workflow_id'],
additionalProperties: false,
},
output: JSON_OUTPUT,
async execute(args, exec) {
try {
const limit = args?.limit ?? 8
const payload = await consoleApi(
'GET',
`/api/workflows/${encodeURIComponent(args.workflow_id)}/runs?limit=${limit}`,
undefined,
{ signal: exec?.signal },
)
return {
ok: true,
workflow_id: payload.workflow_id,
runs: (payload.runs ?? []).map(compactRun),
}
} catch (error) {
return toolErrorResult(error)
}
},
})
register({
name: 'gyxx_workflow_diagnose',
description:
'获取一次运行的完整诊断包:运行状态、步骤明细、journal 追踪路径与脱敏日志尾部。' +
'不传 run_id 时自动选择最近一次失败的运行(无失败则最近一次运行)。',
parameters: {
type: 'object',
properties: {
workflow_id: { type: 'string' },
run_id: { type: 'string', description: '缺省时自动选择最近一次失败运行' },
},
required: ['workflow_id'],
additionalProperties: false,
},
output: JSON_OUTPUT,
async execute(args, exec) {
try {
let runId = args?.run_id
if (!runId) {
const runs = await consoleApi(
'GET',
`/api/workflows/${encodeURIComponent(args.workflow_id)}/runs?limit=8`,
undefined,
{ signal: exec?.signal },
)
const list = runs.runs ?? []
const failed = list.find((run) => run.status === 'failed')
const chosen = failed ?? list[0]
if (!chosen) {
return { ok: true, message: '该工作流还没有任何运行记录', workflow_id: args.workflow_id }
}
runId = chosen.run_id
}
const payload = await consoleApi(
'GET',
`/api/workflows/${encodeURIComponent(args.workflow_id)}/runs/${encodeURIComponent(runId)}/diagnosis`,
undefined,
{ signal: exec?.signal },
)
return { ok: true, ...payload, run: compactRun(payload.run) }
} catch (error) {
return toolErrorResult(error)
}
},
})
register({
name: 'gyxx_workflow_trigger',
description:
'触发工作流运行。默认 execute=false 为试运行(演练,无外部副作用);' +
'正式执行必须 execute=true 且 confirmed=true(只有在用户明确确认后才允许)。',
parameters: {
type: 'object',
properties: {
workflow_id: { type: 'string' },
business_date: { type: 'string', pattern: '^\\d{4}-\\d{2}-\\d{2}$', description: '业务日期 YYYY-MM-DD' },
execute: { type: 'boolean', description: '缺省 false = 试运行' },
confirmed: { type: 'boolean', description: '正式执行确认标记,须先征得用户确认' },
shadow: { type: 'boolean', description: '影子模式(对比旧链路),缺省 false' },
force_refresh: { type: 'boolean', description: '忽略幂等跳过强制重采(需正式执行)' },
},
required: ['workflow_id', 'business_date'],
additionalProperties: false,
},
output: JSON_OUTPUT,
async execute(args, exec) {
const execute = args?.execute === true
const confirmed = args?.confirmed === true
if (execute && !confirmed) {
return {
ok: false,
error:
'正式执行(execute=true)会对外部系统产生真实写入。请先用中文向用户说明将执行的操作与影响,' +
'在用户明确确认后,再以 confirmed=true 重新调用本工具。',
}
}
try {
const payload = await consoleApi(
'POST',
`/api/workflows/${encodeURIComponent(args.workflow_id)}/runs`,
{
business_date: args.business_date,
execute,
shadow: args?.shadow === true,
confirmed,
force_refresh: args?.force_refresh === true,
},
{ signal: exec?.signal },
)
return { ok: true, launched: payload }
} catch (error) {
return toolErrorResult(error)
}
},
})
register({
name: 'gyxx_workflow_cancel',
description: '停止某工作流正在进行的执行:手动运行需要 operation_id;定时运行用 scheduled=true。',
parameters: {
type: 'object',
properties: {
workflow_id: { type: 'string' },
operation_id: { type: 'string', description: '手动运行的操作 ID(active_run.operation_id)' },
scheduled: { type: 'boolean', description: 'true 表示停止调度器正在执行的运行' },
},
required: ['workflow_id'],
additionalProperties: false,
},
output: JSON_OUTPUT,
async execute(args, exec) {
try {
if (args?.scheduled === true) {
const payload = await consoleApi(
'DELETE',
`/api/workflows/${encodeURIComponent(args.workflow_id)}/scheduled-run`,
{},
{ signal: exec?.signal },
)
return { ok: true, result: payload }
}
if (typeof args?.operation_id !== 'string' || !args.operation_id) {
return { ok: false, error: '停止手动运行必须提供 operation_id(见 active_run.operation_id)' }
}
const payload = await consoleApi(
'DELETE',
`/api/workflows/${encodeURIComponent(args.workflow_id)}/runs/${encodeURIComponent(args.operation_id)}`,
{},
{ signal: exec?.signal },
)
return { ok: true, result: payload }
} catch (error) {
return toolErrorResult(error)
}
},
})
register({
name: 'gyxx_schedule_update',
description:
'修改工作流的定时调度(启用/停用、启动时间、周期)。采用读-改-写:' +
'先用 gyxx_workflow_detail 拿到 schedule 与 schedule_revision,再把完整调度对象' +
'与 revision 一并提交。修改前必须向用户说明并获得确认。',
parameters: {
type: 'object',
properties: {
workflow_id: { type: 'string' },
schedule_revision: { type: 'string', description: 'gyxx_workflow_detail 返回的版本号' },
kind: { type: 'string', enum: ['daily', 'weekly', 'monthly', 'interval_days'] },
at: {
description: '启动时间,HH:MM 字符串或字符串数组',
anyOf: [{ type: 'string' }, { type: 'array', items: { type: 'string' }, minItems: 1 }],
},
enabled: { type: 'boolean' },
days: { type: 'array', items: { type: 'string' }, description: 'weekly 时的星期列表' },
day_of_month: { type: 'integer', description: 'monthly 时的日期' },
every_days: { type: 'integer', description: 'interval_days 时的间隔天数' },
business_date_offset_days: { type: 'integer', description: '业务日期偏移,缺省 0' },
},
required: ['workflow_id', 'schedule_revision', 'kind', 'at'],
additionalProperties: false,
},
output: JSON_OUTPUT,
async execute(args, exec) {
try {
const body = {
kind: args.kind,
at: args.at,
enabled: args?.enabled ?? true,
business_date_offset_days: args?.business_date_offset_days ?? 0,
}
for (const key of ['days', 'day_of_month', 'every_days']) {
if (args?.[key] !== undefined) body[key] = args[key]
}
const payload = await consoleApi(
'PUT',
`/api/workflows/${encodeURIComponent(args.workflow_id)}/schedule`,
body,
{ signal: exec?.signal, ifMatch: args.schedule_revision },
)
return { ok: true, result: payload }
} catch (error) {
return toolErrorResult(error)
}
},
})
}
/* ------------------------------------------------------------------ */
/* 提问 / 诊断 / 修复会话 */
/* ------------------------------------------------------------------ */
function buildSessionPrompt(action, workflow, question) {
const label = `${workflow.id}(${workflow.name ?? workflow.id})`
const last = workflow.last_run
const context = last
? `最近一次运行:${last.status},业务日期 ${last.business_date},run_id=${last.run_id}` +
(last.error ? `,错误:${last.error}` : '')
: '该工作流暂无运行记录'
if (action === 'diagnose') {
return (
`【工作台诊断请求】请诊断工作流 ${label}。\n${context}\n` +
'请调用 gyxx_workflow_diagnose 获取最近一次失败运行的诊断包,' +
'按「失败步骤 → 根因推测 → 建议修复 → 风险说明」输出中文诊断报告。'
)
}
if (action === 'repair') {
return (
`【工作台修复请求】请修复工作流 ${label}。\n${context}\n` +
'先用 gyxx_workflow_diagnose 定位失败根因,给出候选修复方案并逐条说明影响;' +
'任何正式执行或调度修改都必须先征得我的确认(遵守安全契约),确认后再执行。'
)
}
return (
`【工作台提问】工作流 ${label}。\n${context}\n` +
`我的问题:${question ?? '这个工作流现在状态如何?'}\n` +
'请结合 gyxx_workflow_detail / gyxx_workflow_runs / gyxx_workflow_diagnose 回答。'
)
}
/* ------------------------------------------------------------------ */
/* 桥接服务(供侧边栏面板使用) */
/* ------------------------------------------------------------------ */
function isLoopbackOrigin(origin) {
if (!origin) return null
try {
const url = new URL(origin)
if (url.protocol !== 'http:' && url.protocol !== 'https:') return null
if (['127.0.0.1', 'localhost', '[::1]'].includes(url.hostname)) return origin
} catch {
return null
}
return null
}
function readJsonBody(req) {
return new Promise((resolve, reject) => {
let size = 0
const chunks = []
req.on('data', (chunk) => {
size += chunk.length
if (size > 64 * 1024) {
reject(new Error('请求正文过大'))
req.destroy()
return
}
chunks.push(chunk)
})
req.on('end', () => {
if (chunks.length === 0) return resolve({})
try {
resolve(JSON.parse(Buffer.concat(chunks).toString('utf8')))
} catch {
reject(new Error('请求 JSON 无效'))
}
})
req.on('error', reject)
})
}
function createBridgeServer(ctx, config, consoleApi, alerts, openChatSession) {
const standalonePage = () => {
try {
return readFileSync(join(PLUGIN_DIR, 'client', 'standalone.html'))
} catch {
return Buffer.from('<h1>gyxx-workbench bridge</h1>', 'utf8')
}
}
const sendJson = (req, res, status, payload) => {
const origin = isLoopbackOrigin(req.headers.origin)
const body = JSON.stringify(payload)
res.writeHead(status, {
'content-type': 'application/json; charset=utf-8',
'cache-control': 'no-store',
...(origin ? { 'access-control-allow-origin': origin, vary: 'origin' } : {}),
})
res.end(body)
}
const server = http.createServer(async (req, res) => {
const url = new URL(req.url ?? '/', `http://${req.headers.host ?? '127.0.0.1'}`)
const path = url.pathname
const origin = isLoopbackOrigin(req.headers.origin)
if (req.method === 'OPTIONS') {
res.writeHead(204, {
...(origin
? {
'access-control-allow-origin': origin,
'access-control-allow-methods': 'GET,POST,OPTIONS',
'access-control-allow-headers': 'content-type,x-gyxx-workbench',
'access-control-max-age': '3600',
vary: 'origin',
}
: {}),
})
res.end()
return
}
try {
if (req.method === 'GET' && path === '/') {
res.writeHead(200, { 'content-type': 'text/html; charset=utf-8' })
res.end(standalonePage())
return
}
if (req.method === 'GET' && path === '/healthz') {
sendJson(req, res, 200, { ok: true, name })
return
}
if (req.method === 'GET' && path === '/bridge/workflows') {
const overview = await consoleApi('GET', '/api/overview')
sendJson(req, res, 200, {
ok: true,
summary: overview.summary ?? null,
generated_at: overview.generated_at ?? null,
workflows: (overview.workflows ?? []).map(compactWorkflow),
})
return
}
const detailMatch = /^\/bridge\/workflows\/([^/]+)$/.exec(path)
if (req.method === 'GET' && detailMatch) {
const payload = await consoleApi(
'GET',
`/api/workflows/${encodeURIComponent(decodeURIComponent(detailMatch[1]))}`,
)
sendJson(req, res, 200, { ok: true, ...compactDetail(payload) })
return
}
const runsMatch = /^\/bridge\/workflows\/([^/]+)\/runs$/.exec(path)
if (req.method === 'GET' && runsMatch) {
const limit = Math.min(Math.max(Number(url.searchParams.get('limit')) || 8, 1), 20)
const payload = await consoleApi(
'GET',
`/api/workflows/${encodeURIComponent(decodeURIComponent(runsMatch[1]))}/runs?limit=${limit}`,
)
sendJson(req, res, 200, {
ok: true,
workflow_id: payload.workflow_id,
runs: (payload.runs ?? []).map(compactRun),
})
return
}
const diagnosisMatch = /^\/bridge\/workflows\/([^/]+)\/diagnosis$/.exec(path)
if (req.method === 'GET' && diagnosisMatch) {
const workflowId = decodeURIComponent(diagnosisMatch[1])
let runId = url.searchParams.get('run_id') || ''
if (!runId) {
const runs = await consoleApi(
'GET',
`/api/workflows/${encodeURIComponent(workflowId)}/runs?limit=8`,
)
const list = runs.runs ?? []
const chosen = list.find((run) => run.status === 'failed') ?? list[0]
if (!chosen) {
sendJson(req, res, 200, { ok: true, message: '该工作流暂无运行记录', logs: [] })
return
}
runId = chosen.run_id
}
const payload = await consoleApi(
'GET',
`/api/workflows/${encodeURIComponent(workflowId)}/runs/${encodeURIComponent(runId)}/diagnosis`,
)
sendJson(req, res, 200, { ok: true, ...payload })
return
}
if (req.method === 'GET' && path === '/bridge/alerts') {
sendJson(req, res, 200, { ok: true, alerts: alerts.list() })
return
}
// —— 以下为写操作:要求自定义头,防跨站表单提交 ——
if (req.method === 'POST' && (path === '/bridge/trigger' || path === '/bridge/cancel' || path === '/bridge/ask' || path === '/bridge/alerts/seen')) {
if (req.headers['x-gyxx-workbench'] !== '1') {
sendJson(req, res, 403, { ok: false, error: '缺少 x-gyxx-workbench 写操作标识' })
return
}
const contentType = String(req.headers['content-type'] ?? '').split(';')[0].trim().toLowerCase()
if (contentType !== 'application/json') {
sendJson(req, res, 415, { ok: false, error: '写操作只接受 application/json' })
return
}
const body = await readJsonBody(req)
if (path === '/bridge/trigger') {
const execute = body.execute === true
const confirmed = body.confirmed === true
if (execute && !confirmed) {
sendJson(req, res, 400, { ok: false, error: '正式执行必须 confirmed=true' })
return
}
const payload = await consoleApi(
'POST',
`/api/workflows/${encodeURIComponent(String(body.workflow_id ?? ''))}/runs`,
{
business_date: String(body.business_date ?? ''),
execute,
shadow: body.shadow === true,
confirmed,
force_refresh: body.force_refresh === true,
},
)
sendJson(req, res, 200, { ok: true, launched: payload })
return
}
if (path === '/bridge/cancel') {
const workflowId = String(body.workflow_id ?? '')
if (body.scheduled === true) {
const payload = await consoleApi(
'DELETE',
`/api/workflows/${encodeURIComponent(workflowId)}/scheduled-run`,
{},
)
sendJson(req, res, 200, { ok: true, result: payload })
return
}
const operationId = String(body.operation_id ?? '')
if (!operationId) {
sendJson(req, res, 400, { ok: false, error: '缺少 operation_id' })
return
}
const payload = await consoleApi(
'DELETE',
`/api/workflows/${encodeURIComponent(workflowId)}/runs/${encodeURIComponent(operationId)}`,
{},
)
sendJson(req, res, 200, { ok: true, result: payload })
return
}
if (path === '/bridge/alerts/seen') {
alerts.markSeen(typeof body.id === 'string' ? body.id : null)
sendJson(req, res, 200, { ok: true })
return
}
// /bridge/ask —— 创建工作流上下文会话
const workflowId = String(body.workflow_id ?? '')
const action = ['ask', 'diagnose', 'repair'].includes(body.action) ? body.action : 'ask'
let workflow
try {
const detail = await consoleApi('GET', `/api/workflows/${encodeURIComponent(workflowId)}`)
workflow = compactWorkflow(detail.workflow ?? {})
} catch {
workflow = { id: workflowId, name: workflowId, last_run: null }
}
const prompt = buildSessionPrompt(action, workflow, typeof body.question === 'string' ? body.question : '')
const sessionId = await openChatSession(prompt)
if (!sessionId) {
sendJson(req, res, 503, {
ok: false,
error: '当前 dsh 组合不支持编程式会话创建,请在对话中直接提问',
})
return
}
sendJson(req, res, 200, { ok: true, session_id: sessionId, action })
return
}
sendJson(req, res, 404, { ok: false, error: '接口不存在' })
} catch (error) {
const status = error instanceof ConsoleApiError && error.status ? error.status : 500
sendJson(req, res, status, {
ok: false,
error: error?.message ?? '桥接服务内部错误',
})
}
})
return server
}
/* ------------------------------------------------------------------ */
/* 插件入口 */
/* ------------------------------------------------------------------ */
export function apply(ctx, rawConfig) {
const config = {
consoleBaseUrl: process.env.GYXX_CONSOLE_URL ?? 'http://127.0.0.1:8765',
consoleToken: process.env.GYXX_CONSOLE_TOKEN ?? '',
bridgeHost: '127.0.0.1',
bridgePort: 8790,
pollIntervalMs: 30_000,
autoDiagnose: false,
projectRoot: process.cwd(),
...(rawConfig && typeof rawConfig === 'object' ? rawConfig : {}),
}
const consoleApi = makeConsoleClient(config)
// 1) 智能体工具
defineWorkflowTools(ctx, consoleApi)
// 2) 系统提示词
ctx.systemPrompt.section({
name: 'gyxx-workbench',
order: 700,
text: SYSTEM_PROMPT_TEXT,
})
// 3) 失败监控器 + 告警缓存
const state = {
seen: new Map(),
items: [],
}
const alerts = {
list: () => state.items.map((item) => ({ ...item })),
markSeen(id) {
if (id === null) state.items.forEach((item) => (item.seen = true))
else {
const target = state.items.find((item) => item.id === id)
if (target) target.seen = true
}
},
push(alert) {
state.items.unshift(alert)
if (state.items.length > 50) state.items.length = 50
},
}
const openChatSession = async (prompt) => {
const agents = typeof ctx.get === 'function' ? ctx.get('agents') : ctx.agents
if (!agents || typeof agents.create !== 'function') return null
const sessionId = `gyxx-wb-${Date.now().toString(36)}-${randomUUID().slice(0, 8)}`
const handle = await agents.create({
sessionId,
meta: { cwd: config.projectRoot, origin: 'gyxx-workbench', isSeeded: true },
})
handle.agent.followup({
id: randomUUID(),
role: 'user',
content: [{ type: 'text', text: prompt }],
source: { kind: 'user' },
})
return sessionId
}
ctx.effect(() => {
let stopped = false
let timer = null
const poll = async () => {
try {
const overview = await consoleApi('GET', '/api/overview')
for (const item of overview.workflows ?? []) {
const last = item.last_run
if (!last || last.status !== 'failed') {
if (last && last.status !== 'failed') state.seen.delete(item.id)
continue
}
if (state.seen.get(item.id) === last.run_id) continue
state.seen.set(item.id, last.run_id)
const alert = {
id: randomUUID(),
kind: 'workflow-failed',
workflow_id: item.id,
name: item.name ?? item.id,
run_id: last.run_id,
business_date: last.business_date ?? null,
error: last.error ?? null,
at: new Date().toISOString(),
seen: false,
}
alerts.push(alert)
try {
ctx.emit('gyxx-workbench/alert', alert)
} catch {
// 自定义事件在某些组合上不可用时忽略
}
if (config.autoDiagnose) {
const prompt = buildSessionPrompt('diagnose', compactWorkflow(item), '')
openChatSession(prompt)?.catch(() => {})
}
}
} catch {
// 控制台离线时静默,下一轮继续
}
if (!stopped) timer = setTimeout(poll, config.pollIntervalMs)
}
timer = setTimeout(poll, 5_000)
return () => {
stopped = true
if (timer) clearTimeout(timer)
}
}, 'gyxx-workbench: monitor')
// 4) 桥接服务(侧边栏面板数据 + 会话创建 + 降级页面)
ctx.effect(async () => {
const server = createBridgeServer(ctx, config, consoleApi, alerts, openChatSession)
try {
await new Promise((resolve, reject) => {
server.once('error', reject)
server.listen(config.bridgePort, config.bridgeHost, resolve)
})
} catch (error) {
// 端口占用等问题不应阻断插件加载(工具与监控仍可用)
ctx.logger?.warn?.(
`[gyxx-workbench] 桥接服务启动失败(${error?.message ?? error}),侧边栏面板不可用`,
)
try {
server.close()
} catch {
// 忽略关闭异常
}
return () => {}
}
const address = server.address()
ctx.logger?.info?.(
`[gyxx-workbench] 桥接服务已启动: http://${config.bridgeHost}:${address?.port ?? config.bridgePort}/`,
)
return () =>
new Promise((resolve) => {
server.close(() => resolve())
})
}, 'gyxx-workbench: bridge')
}
-580
View File
@@ -1,580 +0,0 @@
window.__ModuleLoader__.load({ id: "@gyxx/dsh-plugin-gyxx-workbench", factory: (require) => {
var module = { exports: {} };
var exports = module.exports;
var __defProp = Object.defineProperty;
var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
var __getOwnPropNames = Object.getOwnPropertyNames;
var __hasOwnProp = Object.prototype.hasOwnProperty;
var __export = (target, all) => {
for (var name in all)
__defProp(target, name, { get: all[name], enumerable: true });
};
var __copyProps = (to, from, except, desc) => {
if (from && typeof from === "object" || typeof from === "function") {
for (let key of __getOwnPropNames(from))
if (!__hasOwnProp.call(to, key) && key !== except)
__defProp(to, key, { get: () => from[key], enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable });
}
return to;
};
var __toCommonJS = (mod) => __copyProps(__defProp({}, "__esModule", { value: true }), mod);
// client/src/index.jsx
var index_exports = {};
__export(index_exports, {
apply: () => apply,
inject: () => inject
});
module.exports = __toCommonJS(index_exports);
var import_react4 = require("react");
// client/src/Panel.jsx
var import_react = require("react");
// client/src/store.js
var listeners = /* @__PURE__ */ new Set();
var state = {
open: false,
loading: false,
error: null,
generatedAt: null,
summary: null,
workflows: [],
selectedId: null,
detail: null,
detailLoading: false,
runs: [],
diagnosis: null,
diagnosisLoading: false,
question: "",
businessDate: defaultBusinessDate(),
alerts: [],
seenAlertIds: {},
toasts: []
};
function defaultBusinessDate() {
const now = /* @__PURE__ */ new Date();
now.setDate(now.getDate() - 1);
const pad = (value) => String(value).padStart(2, "0");
return `${now.getFullYear()}-${pad(now.getMonth() + 1)}-${pad(now.getDate())}`;
}
function getState() {
return state;
}
function update(patch) {
state = { ...state, ...patch };
for (const listener of listeners) listener();
}
function subscribe(listener) {
listeners.add(listener);
return () => listeners.delete(listener);
}
function pushToast(toast) {
const id = `t-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 6)}`;
update({ toasts: [...state.toasts, { id, ...toast }] });
return id;
}
function dismissToast(id) {
update({ toasts: state.toasts.filter((toast) => toast.id !== id) });
}
// client/src/api.js
function bridgeBase() {
if (typeof window !== "undefined") {
if (window.__GYXX_WB_BRIDGE__) return window.__GYXX_WB_BRIDGE__;
try {
const saved = window.localStorage?.getItem("gyxx.wb.bridge");
if (saved) return saved;
} catch {
}
}
return "http://127.0.0.1:8790";
}
async function request(method, path, body) {
const response = await fetch(`${bridgeBase()}${path}`, {
method,
headers: {
accept: "application/json",
...body !== void 0 ? { "content-type": "application/json", "x-gyxx-workbench": "1" } : {}
},
body: body === void 0 ? void 0 : JSON.stringify(body)
});
let payload;
try {
payload = await response.json();
} catch {
payload = { ok: false, error: `\u6865\u63A5\u670D\u52A1\u8FD4\u56DE\u4E86\u975E JSON \u5185\u5BB9\uFF08HTTP ${response.status}\uFF09` };
}
if (!response.ok) {
throw new Error(payload?.error ?? `\u6865\u63A5\u670D\u52A1\u9519\u8BEF\uFF08HTTP ${response.status}\uFF09`);
}
return payload;
}
var api = {
listWorkflows: () => request("GET", "/bridge/workflows"),
workflowDetail: (workflowId) => request("GET", `/bridge/workflows/${encodeURIComponent(workflowId)}`),
workflowRuns: (workflowId, limit = 6) => request("GET", `/bridge/workflows/${encodeURIComponent(workflowId)}/runs?limit=${limit}`),
diagnosis: (workflowId, runId) => request(
"GET",
`/bridge/workflows/${encodeURIComponent(workflowId)}/diagnosis${runId ? `?run_id=${encodeURIComponent(runId)}` : ""}`
),
alerts: () => request("GET", "/bridge/alerts"),
markAlertSeen: (id) => request("POST", "/bridge/alerts/seen", { id }),
trigger: (payload) => request("POST", "/bridge/trigger", payload),
cancel: (payload) => request("POST", "/bridge/cancel", payload),
ask: (payload) => request("POST", "/bridge/ask", payload)
};
// client/src/Panel.jsx
var import_jsx_runtime = require("react/jsx-runtime");
function useWorkbench() {
return (0, import_react.useSyncExternalStore)(subscribe, getState);
}
var STATUS_LABELS = {
success: "\u6210\u529F",
failed: "\u5931\u8D25",
running: "\u8FD0\u884C\u4E2D",
cancelled: "\u5DF2\u505C\u6B62",
none: "\u672A\u8FD0\u884C"
};
function statusOf(workflow) {
if (workflow.active_run) return "running";
return workflow.last_run?.status ?? "none";
}
function formatTime(value) {
if (!value) return "\u2014";
const date = new Date(value);
if (Number.isNaN(date.getTime())) return String(value);
const pad = (n) => String(n).padStart(2, "0");
return `${pad(date.getMonth() + 1)}-${pad(date.getDate())} ${pad(date.getHours())}:${pad(date.getMinutes())}`;
}
function formatDuration(seconds) {
if (seconds == null) return "";
if (seconds < 60) return `${seconds}s`;
return `${Math.floor(seconds / 60)}m${seconds % 60 ? `${seconds % 60}s` : ""}`;
}
async function refreshWorkflows() {
update({ loading: true, error: null });
try {
const payload = await api.listWorkflows();
update({
loading: false,
workflows: payload.workflows ?? [],
summary: payload.summary ?? null,
generatedAt: payload.generated_at ?? null
});
} catch (error) {
update({ loading: false, error: error.message });
}
}
async function selectWorkflow(workflowId) {
update({ selectedId: workflowId, detailLoading: true, diagnosis: null, runs: [] });
try {
const [detail, runs] = await Promise.all([
api.workflowDetail(workflowId).catch(() => null),
api.workflowRuns(workflowId, 6).catch(() => ({ runs: [] }))
]);
update({ detailLoading: false, detail, runs: runs.runs ?? [] });
} catch (error) {
update({ detailLoading: false, error: error.message });
}
}
async function loadDiagnosis(workflowId, runId) {
update({ diagnosisLoading: true });
try {
const payload = await api.diagnosis(workflowId, runId);
update({ diagnosisLoading: false, diagnosis: payload });
} catch (error) {
update({ diagnosisLoading: false });
pushToast({ kind: "error", text: `\u8BCA\u65AD\u6570\u636E\u52A0\u8F7D\u5931\u8D25\uFF1A${error.message}` });
}
}
async function doAsk(action) {
const s = getState();
if (!s.selectedId) return;
const question = s.question.trim();
try {
const payload = await api.ask({
workflow_id: s.selectedId,
action,
question
});
const label = { ask: "\u63D0\u95EE", diagnose: "\u8BCA\u65AD", repair: "\u4FEE\u590D" }[action] ?? "\u4F1A\u8BDD";
pushToast({
kind: "ok",
text: `\u5DF2\u521B\u5EFA${label}\u4F1A\u8BDD\uFF08${String(payload.session_id).slice(0, 18)}\u2026\uFF09\uFF0C\u8BF7\u5728\u5DE6\u4FA7\u4F1A\u8BDD\u5217\u8868\u4E2D\u67E5\u770B`
});
update({ question: "" });
} catch (error) {
pushToast({ kind: "error", text: `\u521B\u5EFA\u4F1A\u8BDD\u5931\u8D25\uFF1A${error.message}` });
}
}
async function doTrigger(execute) {
const s = getState();
if (!s.selectedId) return;
if (execute) {
const ok = window.confirm(
`\u6B63\u5F0F\u6267\u884C ${s.selectedId}\uFF08\u4E1A\u52A1\u65E5\u671F ${s.businessDate}\uFF09\u4F1A\u5BF9\u5916\u90E8\u7CFB\u7EDF\u4EA7\u751F\u771F\u5B9E\u5199\u5165\u3002\u786E\u8BA4\u7EE7\u7EED\uFF1F`
);
if (!ok) return;
}
try {
await api.trigger({
workflow_id: s.selectedId,
business_date: s.businessDate,
execute,
confirmed: execute
});
pushToast({ kind: "ok", text: execute ? "\u5DF2\u53D1\u8D77\u6B63\u5F0F\u6267\u884C" : "\u5DF2\u53D1\u8D77\u8BD5\u8FD0\u884C\uFF08\u65E0\u5916\u90E8\u526F\u4F5C\u7528\uFF09" });
setTimeout(refreshWorkflows, 1500);
} catch (error) {
pushToast({ kind: "error", text: `\u542F\u52A8\u5931\u8D25\uFF1A${error.message}` });
}
}
async function doCancel() {
const s = getState();
const active = s.detail?.workflow?.active_run;
if (!s.selectedId) return;
try {
if (active?.operation_id) {
await api.cancel({ workflow_id: s.selectedId, operation_id: active.operation_id });
} else {
await api.cancel({ workflow_id: s.selectedId, scheduled: true });
}
pushToast({ kind: "ok", text: "\u5DF2\u53D1\u9001\u505C\u6B62\u6307\u4EE4" });
setTimeout(refreshWorkflows, 1500);
} catch (error) {
pushToast({ kind: "error", text: `\u505C\u6B62\u5931\u8D25\uFF1A${error.message}` });
}
}
function SummaryPills({ summary, workflows }) {
const failed = workflows.filter((w) => statusOf(w) === "failed").length;
const running = workflows.filter((w) => w.active_run).length;
return /* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-pills", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("span", { className: "gyxxwb-pill", children: [
"\u5171 ",
summary?.total ?? workflows.length
] }),
running > 0 && /* @__PURE__ */ (0, import_jsx_runtime.jsxs)("span", { className: "gyxxwb-pill gyxxwb-pill-running", children: [
"\u8FD0\u884C\u4E2D ",
running
] }),
failed > 0 && /* @__PURE__ */ (0, import_jsx_runtime.jsxs)("span", { className: "gyxxwb-pill gyxxwb-pill-failed", children: [
"\u5931\u8D25 ",
failed
] })
] });
}
function WorkflowRow({ workflow, selected, onSelect }) {
const status = statusOf(workflow);
return /* @__PURE__ */ (0, import_jsx_runtime.jsxs)(
"button",
{
type: "button",
className: `gyxxwb-row ${selected ? "gyxxwb-row-selected" : ""}`,
onClick: () => onSelect(workflow.id),
children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { className: `gyxxwb-dot gyxxwb-dot-${status}` }),
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("span", { className: "gyxxwb-row-main", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { className: "gyxxwb-row-name", children: workflow.name }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { className: "gyxxwb-row-id", children: workflow.id })
] }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { className: "gyxxwb-row-meta", children: workflow.last_run ? formatTime(workflow.last_run.ended_at ?? workflow.last_run.started_at) : "\u2014" })
]
}
);
}
function RunList({ runs, onDiagnose }) {
if (!runs.length) return /* @__PURE__ */ (0, import_jsx_runtime.jsx)("div", { className: "gyxxwb-empty", children: "\u6682\u65E0\u8FD0\u884C\u8BB0\u5F55" });
return /* @__PURE__ */ (0, import_jsx_runtime.jsx)("div", { className: "gyxxwb-runs", children: runs.map((run) => /* @__PURE__ */ (0, import_jsx_runtime.jsxs)(
"button",
{
type: "button",
className: "gyxxwb-run",
onClick: () => onDiagnose(run.run_id),
title: "\u70B9\u51FB\u67E5\u770B\u8BCA\u65AD\u8BE6\u60C5",
children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { className: `gyxxwb-dot gyxxwb-dot-${run.status}` }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { children: run.business_date }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { className: "gyxxwb-run-mode", children: run.mode === "execute" ? "\u6B63\u5F0F" : "\u8BD5\u8FD0\u884C" }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { children: formatDuration(run.duration_seconds) }),
run.error && /* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { className: "gyxxwb-run-error", children: run.error.slice(0, 60) })
]
},
run.run_id
)) });
}
function DiagnosisView({ diagnosis, loading }) {
if (loading) return /* @__PURE__ */ (0, import_jsx_runtime.jsx)("div", { className: "gyxxwb-empty", children: "\u6B63\u5728\u52A0\u8F7D\u8BCA\u65AD\u6570\u636E\u2026" });
if (!diagnosis) return null;
if (diagnosis.message) return /* @__PURE__ */ (0, import_jsx_runtime.jsx)("div", { className: "gyxxwb-empty", children: diagnosis.message });
const run = diagnosis.run ?? {};
const steps = Array.isArray(run.steps) ? run.steps : [];
const logs = Array.isArray(diagnosis.logs) ? diagnosis.logs : [];
return /* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-diagnosis", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-diagnosis-head", children: [
"\u8BCA\u65AD\uFF1A",
run.run_id,
"\uFF08",
run.status,
"\uFF09"
] }),
steps.filter((step) => step.status === "failed").map((step) => /* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-step-fail", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-step-id", children: [
"\u2717 ",
step.id,
"\uFF08\u9000\u51FA\u7801 ",
step.exit_code ?? "\u2014",
"\uFF09"
] }),
step.error && /* @__PURE__ */ (0, import_jsx_runtime.jsx)("pre", { className: "gyxxwb-pre", children: step.error })
] }, step.id)),
logs.map((log) => /* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-log", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("div", { className: "gyxxwb-log-path", children: log.path }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("pre", { className: "gyxxwb-pre", children: log.tail })
] }, log.path)),
!steps.some((s) => s.status === "failed") && !logs.length && /* @__PURE__ */ (0, import_jsx_runtime.jsx)("div", { className: "gyxxwb-empty", children: "\u8BE5\u8FD0\u884C\u6CA1\u6709\u5931\u8D25\u6B65\u9AA4\u6216\u65E5\u5FD7" })
] });
}
function WorkflowPanel() {
const s = useWorkbench();
(0, import_react.useEffect)(() => {
if (!s.open) return void 0;
refreshWorkflows();
const timer = setInterval(refreshWorkflows, 1e4);
return () => clearInterval(timer);
}, [s.open]);
const onSelect = (0, import_react.useCallback)((workflowId) => {
selectWorkflow(workflowId);
}, []);
const groups = [];
const byModule = /* @__PURE__ */ new Map();
for (const workflow of s.workflows) {
const key = workflow.module_label ?? workflow.module;
if (!byModule.has(key)) byModule.set(key, []);
byModule.get(key).push(workflow);
}
for (const [label, items] of byModule) groups.push({ label, items });
const selected = s.workflows.find((w) => w.id === s.selectedId) ?? null;
return /* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: `gyxxwb-panel ${s.open ? "gyxxwb-panel-open" : ""}`, children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-panel-head", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { className: "gyxxwb-title", children: "GYXX \u5DE5\u4F5C\u6D41" }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)(SummaryPills, { summary: s.summary, workflows: s.workflows }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("button", { type: "button", className: "gyxxwb-icon-btn", title: "\u5237\u65B0", onClick: refreshWorkflows, children: "\u27F3" }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)(
"button",
{
type: "button",
className: "gyxxwb-icon-btn",
title: "\u5173\u95ED",
onClick: () => update({ open: false }),
children: "\u2715"
}
)
] }),
s.error && /* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-error", children: [
"\u65E0\u6CD5\u8FDE\u63A5\u5DE5\u4F5C\u53F0\u6865\u63A5\u670D\u52A1\uFF1A",
s.error,
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("br", {}),
"\u8BF7\u786E\u8BA4 dsh \u5DF2\u901A\u8FC7 workbench/cordis.yml \u542F\u52A8\u3002"
] }),
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-panel-body", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-list", children: [
groups.map((group) => /* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-group", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("div", { className: "gyxxwb-group-label", children: group.label }),
group.items.map((workflow) => /* @__PURE__ */ (0, import_jsx_runtime.jsx)(
WorkflowRow,
{
workflow,
selected: workflow.id === s.selectedId,
onSelect
},
workflow.id
))
] }, group.label)),
!s.loading && !s.workflows.length && !s.error && /* @__PURE__ */ (0, import_jsx_runtime.jsx)("div", { className: "gyxxwb-empty", children: "\u672A\u53D1\u73B0\u5DE5\u4F5C\u6D41" })
] }),
selected && /* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-detail", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-detail-title", children: [
selected.name,
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("span", { className: `gyxxwb-status gyxxwb-status-${statusOf(selected)}`, children: STATUS_LABELS[statusOf(selected)] })
] }),
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-detail-sub", children: [
selected.id,
" \xB7 \u8C03\u5EA6",
" ",
selected.schedule ? `${selected.schedule.enabled ? "\u542F\u7528" : "\u505C\u7528"} ${Array.isArray(selected.schedule.at) ? selected.schedule.at.join("/") : selected.schedule.at ?? ""}` : "\u65E0",
selected.next_run_at ? ` \xB7 \u4E0B\u6B21 ${formatTime(selected.next_run_at)}` : ""
] }),
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-actions", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("button", { type: "button", onClick: () => doAsk("diagnose"), children: "\u8BCA\u65AD" }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("button", { type: "button", onClick: () => doAsk("repair"), children: "\u4FEE\u590D" }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("button", { type: "button", onClick: () => doTrigger(false), children: "\u8BD5\u8FD0\u884C" }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("button", { type: "button", className: "gyxxwb-danger", onClick: () => doTrigger(true), children: "\u6B63\u5F0F\u8FD0\u884C" }),
selected.active_run && /* @__PURE__ */ (0, import_jsx_runtime.jsx)("button", { type: "button", className: "gyxxwb-danger", onClick: doCancel, children: "\u505C\u6B62" })
] }),
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-ask", children: [
/* @__PURE__ */ (0, import_jsx_runtime.jsx)(
"input",
{
value: s.question,
placeholder: "\u5C31\u8FD9\u4E2A\u5DE5\u4F5C\u6D41\u63D0\u95EE\uFF0C\u4F8B\u5982\uFF1A\u6628\u5929\u4E3A\u4EC0\u4E48\u5931\u8D25\uFF1F",
onChange: (event) => update({ question: event.target.value }),
onKeyDown: (event) => {
if (event.key === "Enter") doAsk("ask");
}
}
),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("button", { type: "button", onClick: () => doAsk("ask"), children: "\u63D0\u95EE" })
] }),
/* @__PURE__ */ (0, import_jsx_runtime.jsxs)("div", { className: "gyxxwb-date-row", children: [
"\u4E1A\u52A1\u65E5\u671F",
" ",
/* @__PURE__ */ (0, import_jsx_runtime.jsx)(
"input",
{
value: s.businessDate,
onChange: (event) => update({ businessDate: event.target.value }),
pattern: "\\d{4}-\\d{2}-\\d{2}"
}
)
] }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)("div", { className: "gyxxwb-section-label", children: "\u6700\u8FD1\u8FD0\u884C" }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)(RunList, { runs: s.runs, onDiagnose: (runId) => loadDiagnosis(selected.id, runId) }),
/* @__PURE__ */ (0, import_jsx_runtime.jsx)(DiagnosisView, { diagnosis: s.diagnosis, loading: s.diagnosisLoading })
] })
] })
] });
}
// client/src/Toasts.jsx
var import_react2 = require("react");
var import_react3 = require("react");
var import_jsx_runtime2 = require("react/jsx-runtime");
function useWorkbench2() {
return (0, import_react3.useSyncExternalStore)(subscribe, getState);
}
function Toasts() {
const s = useWorkbench2();
(0, import_react2.useEffect)(() => {
let stopped = false;
let timer = null;
const poll = async () => {
try {
const payload = await api.alerts();
const fresh = (payload.alerts ?? []).filter(
(alert) => !alert.seen && !getState().seenAlertIds[alert.id]
);
if (fresh.length) {
const seen = { ...getState().seenAlertIds };
for (const alert of fresh) {
seen[alert.id] = true;
pushToast({ kind: "failed", text: `\u5DE5\u4F5C\u6D41\u300C${alert.name}\u300D\u8FD0\u884C\u5931\u8D25`, alert });
}
update({ seenAlertIds: seen });
}
} catch {
}
if (!stopped) timer = setTimeout(poll, 3e4);
};
timer = setTimeout(poll, 8e3);
return () => {
stopped = true;
if (timer) clearTimeout(timer);
};
}, []);
if (!s.toasts.length) return null;
return /* @__PURE__ */ (0, import_jsx_runtime2.jsx)("div", { className: "gyxxwb-toasts", children: s.toasts.map((toast) => /* @__PURE__ */ (0, import_jsx_runtime2.jsxs)("div", { className: `gyxxwb-toast gyxxwb-toast-${toast.kind}`, children: [
/* @__PURE__ */ (0, import_jsx_runtime2.jsx)("span", { className: "gyxxwb-toast-text", children: toast.text }),
toast.alert && /* @__PURE__ */ (0, import_jsx_runtime2.jsx)(
"button",
{
type: "button",
onClick: async () => {
try {
await api.ask({ workflow_id: toast.alert.workflow_id, action: "diagnose" });
await api.markAlertSeen(toast.alert.id).catch(() => {
});
pushToast({ kind: "ok", text: "\u5DF2\u521B\u5EFA\u8BCA\u65AD\u4F1A\u8BDD\uFF0C\u8BF7\u5728\u5DE6\u4FA7\u4F1A\u8BDD\u5217\u8868\u67E5\u770B" });
} catch (error) {
pushToast({ kind: "error", text: `\u521B\u5EFA\u4F1A\u8BDD\u5931\u8D25\uFF1A${error.message}` });
}
dismissToast(toast.id);
},
children: "\u7ACB\u5373\u8BCA\u65AD"
}
),
/* @__PURE__ */ (0, import_jsx_runtime2.jsx)("button", { type: "button", title: "\u5173\u95ED", onClick: () => dismissToast(toast.id), children: "\u2715" })
] }, toast.id)) });
}
// client/src/styles.css
var styles_default = "/* GYXX \u5DE5\u4F5C\u53F0\u9762\u677F\u6837\u5F0F \u2014\u2014 \u4EE5\u6587\u672C\u5F62\u5F0F\u5185\u8054\u8FDB\u5BA2\u6237\u7AEF\u5305\uFF0C\u8FD0\u884C\u65F6\u6CE8\u5165 <style>\u3002 */\n\n.gyxxwb-overlay {\n position: fixed;\n inset: 0;\n pointer-events: none;\n z-index: 70;\n font-family:\n -apple-system, BlinkMacSystemFont, 'Segoe UI', 'PingFang SC',\n 'Hiragino Sans GB', 'Microsoft YaHei', sans-serif;\n}\n\n.gyxxwb-panel {\n position: absolute;\n top: 0;\n bottom: 0;\n left: 264px;\n width: 380px;\n display: none;\n flex-direction: column;\n background: #ffffff;\n border-right: 1px solid #e4e4e7;\n box-shadow: 8px 0 24px rgba(0, 0, 0, 0.08);\n pointer-events: auto;\n color: #18181b;\n}\n\n.gyxxwb-panel-open {\n display: flex;\n}\n\n.gyxxwb-panel-head {\n display: flex;\n align-items: center;\n gap: 8px;\n padding: 10px 12px;\n border-bottom: 1px solid #e4e4e7;\n}\n\n.gyxxwb-title {\n font-weight: 600;\n font-size: 14px;\n}\n\n.gyxxwb-pills {\n display: flex;\n gap: 4px;\n flex: 1;\n}\n\n.gyxxwb-pill {\n font-size: 11px;\n padding: 1px 8px;\n border-radius: 999px;\n background: #f4f4f5;\n color: #52525b;\n}\n\n.gyxxwb-pill-failed {\n background: #fef2f2;\n color: #b91c1c;\n}\n\n.gyxxwb-pill-running {\n background: #eff6ff;\n color: #1d4ed8;\n}\n\n.gyxxwb-icon-btn {\n border: none;\n background: transparent;\n cursor: pointer;\n font-size: 14px;\n color: #71717a;\n padding: 4px 6px;\n border-radius: 6px;\n}\n\n.gyxxwb-icon-btn:hover {\n background: #f4f4f5;\n}\n\n.gyxxwb-error {\n margin: 10px 12px;\n padding: 10px;\n border-radius: 8px;\n background: #fef2f2;\n color: #b91c1c;\n font-size: 12px;\n line-height: 1.6;\n}\n\n.gyxxwb-panel-body {\n flex: 1;\n overflow-y: auto;\n display: flex;\n flex-direction: column;\n}\n\n.gyxxwb-group-label {\n padding: 10px 12px 4px;\n font-size: 12px;\n color: #71717a;\n}\n\n.gyxxwb-row {\n display: flex;\n align-items: center;\n gap: 8px;\n width: 100%;\n padding: 7px 12px;\n border: none;\n background: transparent;\n cursor: pointer;\n text-align: left;\n}\n\n.gyxxwb-row:hover {\n background: #f4f4f5;\n}\n\n.gyxxwb-row-selected {\n background: #eef2ff;\n}\n\n.gyxxwb-row-main {\n flex: 1;\n min-width: 0;\n display: flex;\n flex-direction: column;\n}\n\n.gyxxwb-row-name {\n font-size: 13px;\n white-space: nowrap;\n overflow: hidden;\n text-overflow: ellipsis;\n}\n\n.gyxxwb-row-id {\n font-size: 11px;\n color: #a1a1aa;\n}\n\n.gyxxwb-row-meta {\n font-size: 11px;\n color: #71717a;\n}\n\n.gyxxwb-dot {\n width: 8px;\n height: 8px;\n border-radius: 50%;\n background: #d4d4d8;\n flex: none;\n}\n\n.gyxxwb-dot-success {\n background: #16a34a;\n}\n\n.gyxxwb-dot-failed {\n background: #dc2626;\n}\n\n.gyxxwb-dot-running {\n background: #2563eb;\n animation: gyxxwb-pulse 1.2s ease-in-out infinite;\n}\n\n.gyxxwb-dot-cancelled,\n.gyxxwb-dot-none {\n background: #a1a1aa;\n}\n\n@keyframes gyxxwb-pulse {\n 50% {\n opacity: 0.35;\n }\n}\n\n.gyxxwb-detail {\n border-top: 1px solid #e4e4e7;\n padding: 12px;\n background: #fafafa;\n}\n\n.gyxxwb-detail-title {\n font-size: 14px;\n font-weight: 600;\n display: flex;\n align-items: center;\n gap: 8px;\n}\n\n.gyxxwb-status {\n font-size: 11px;\n padding: 1px 8px;\n border-radius: 999px;\n background: #f4f4f5;\n}\n\n.gyxxwb-status-failed {\n background: #fef2f2;\n color: #b91c1c;\n}\n\n.gyxxwb-status-running {\n background: #eff6ff;\n color: #1d4ed8;\n}\n\n.gyxxwb-status-success {\n background: #f0fdf4;\n color: #15803d;\n}\n\n.gyxxwb-detail-sub {\n font-size: 11px;\n color: #71717a;\n margin: 4px 0 10px;\n}\n\n.gyxxwb-actions {\n display: flex;\n flex-wrap: wrap;\n gap: 6px;\n}\n\n.gyxxwb-actions button,\n.gyxxwb-ask button {\n border: 1px solid #e4e4e7;\n background: #ffffff;\n border-radius: 6px;\n font-size: 12px;\n padding: 4px 10px;\n cursor: pointer;\n}\n\n.gyxxwb-actions button:hover,\n.gyxxwb-ask button:hover {\n background: #f4f4f5;\n}\n\n.gyxxwb-actions .gyxxwb-danger {\n border-color: #fecaca;\n color: #b91c1c;\n}\n\n.gyxxwb-ask {\n display: flex;\n gap: 6px;\n margin-top: 8px;\n}\n\n.gyxxwb-ask input {\n flex: 1;\n border: 1px solid #e4e4e7;\n border-radius: 6px;\n padding: 5px 8px;\n font-size: 12px;\n}\n\n.gyxxwb-date-row {\n margin-top: 8px;\n font-size: 11px;\n color: #71717a;\n display: flex;\n align-items: center;\n gap: 6px;\n}\n\n.gyxxwb-date-row input {\n border: 1px solid #e4e4e7;\n border-radius: 6px;\n padding: 3px 6px;\n font-size: 11px;\n width: 110px;\n}\n\n.gyxxwb-section-label {\n margin: 12px 0 4px;\n font-size: 12px;\n color: #71717a;\n}\n\n.gyxxwb-run {\n display: flex;\n align-items: center;\n gap: 8px;\n width: 100%;\n padding: 6px 8px;\n border: none;\n background: transparent;\n cursor: pointer;\n font-size: 12px;\n text-align: left;\n border-radius: 6px;\n}\n\n.gyxxwb-run:hover {\n background: #f4f4f5;\n}\n\n.gyxxwLine truncated
// client/src/index.jsx
var inject = ["slots"];
function useWorkbench3() {
return (0, import_react4.useSyncExternalStore)(subscribe, getState);
}
function injectStyles() {
if (typeof document === "undefined") return;
if (document.getElementById("gyxxwb-styles")) return;
const style = document.createElement("style");
style.id = "gyxxwb-styles";
style.textContent = styles_default;
document.head.appendChild(style);
}
function WorkbenchFooterAction() {
const s = useWorkbench3();
const failed = s.workflows.filter((w) => !w.active_run && w.last_run?.status === "failed").length;
const running = s.workflows.filter((w) => w.active_run).length;
const tone = failed > 0 ? "failed" : running > 0 ? "running" : "success";
return (0, import_react4.createElement)(
"button",
{
type: "button",
className: `gyxxwb-footer-btn${s.open ? " gyxxwb-footer-btn-active" : ""}`,
title: "GYXX \u5DE5\u4F5C\u6D41\u5DE5\u4F5C\u53F0",
onClick: () => {
const next = !getState().open;
update({ open: next });
if (next) refreshWorkflows();
}
},
(0, import_react4.createElement)("span", { className: `gyxxwb-dot gyxxwb-dot-${tone}` }),
(0, import_react4.createElement)("span", null, "\u5DE5\u4F5C\u6D41")
);
}
function WorkbenchOverlay() {
return (0, import_react4.createElement)(
"div",
{ className: "gyxxwb-overlay" },
(0, import_react4.createElement)(WorkflowPanel, null),
(0, import_react4.createElement)(Toasts, null)
);
}
function apply(ctx) {
injectStyles();
ctx.effect(
() => ctx.slots.inject(
"sidebar.footer.action",
() => ctx.slots.register(
{ name: "sidebar.footer.action", id: "gyxx-workbench.footer" },
WorkbenchFooterAction
)
),
"gyxx-workbench: footer action"
);
ctx.effect(
() => ctx.slots.inject(
"shell.overlay",
() => ctx.slots.register(
{ name: "shell.overlay", id: "gyxx-workbench.overlay" },
WorkbenchOverlay
)
),
"gyxx-workbench: overlay"
);
}
return module.exports; } });
//# sourceMappingURL=client.js.map
-7
View File
@@ -1,7 +0,0 @@
{
"version": 3,
"sources": ["../client/src/index.jsx", "../client/src/Panel.jsx", "../client/src/store.js", "../client/src/api.js", "../client/src/Toasts.jsx", "../client/src/styles.css"],
"sourcesContent": ["/**\n * GYXX \u5DE5\u4F5C\u53F0\u5BA2\u6237\u7AEF\u63D2\u4EF6\u5165\u53E3\u3002\n * \u7531 esbuild \u6253\u5305\u4E3A CJS \u5E76\u901A\u8FC7 window.__ModuleLoader__ \u6CE8\u518C\uFF08\u89C1 scripts/build-client.mjs\uFF09\u3002\n * \u8FD0\u884C\u65F6\u53EA\u4F9D\u8D56 dsh \u5E73\u53F0\u5171\u4EAB\u6A21\u5757\u8868\u4E2D\u7684 react\u3002\n */\n\nimport { createElement, useSyncExternalStore } from 'react'\nimport { WorkflowPanel, refreshWorkflows } from './Panel.jsx'\nimport { Toasts } from './Toasts.jsx'\nimport { getState, subscribe, update } from './store.js'\nimport stylesText from './styles.css'\n\nexport const inject = ['slots']\n\nfunction useWorkbench() {\n return useSyncExternalStore(subscribe, getState)\n}\n\nfunction injectStyles() {\n if (typeof document === 'undefined') return\n if (document.getElementById('gyxxwb-styles')) return\n const style = document.createElement('style')\n style.id = 'gyxxwb-styles'\n style.textContent = stylesText\n document.head.appendChild(style)\n}\n\n/** \u4FA7\u8FB9\u680F\u5E95\u90E8\u52A8\u4F5C\uFF1A\u5DE5\u4F5C\u6D41\u9762\u677F\u5F00\u5173 + \u805A\u5408\u72B6\u6001\u70B9\u3002 */\nfunction WorkbenchFooterAction() {\n const s = useWorkbench()\n const failed = s.workflows.filter((w) => !w.active_run && w.last_run?.status === 'failed').length\n const running = s.workflows.filter((w) => w.active_run).length\n const tone = failed > 0 ? 'failed' : running > 0 ? 'running' : 'success'\n return createElement(\n 'button',\n {\n type: 'button',\n className: `gyxxwb-footer-btn${s.open ? ' gyxxwb-footer-btn-active' : ''}`,\n title: 'GYXX \u5DE5\u4F5C\u6D41\u5DE5\u4F5C\u53F0',\n onClick: () => {\n const next = !getState().open\n update({ open: next })\n if (next) refreshWorkflows()\n },\n },\n createElement('span', { className: `gyxxwb-dot gyxxwb-dot-${tone}` }),\n createElement('span', null, '\u5DE5\u4F5C\u6D41'),\n )\n}\n\n/** \u8986\u76D6\u5C42\u6839\uFF1A\u5DE6\u4FA7\u62BD\u5C49\u9762\u677F + \u5931\u8D25 toast\u3002 */\nfunction WorkbenchOverlay() {\n return createElement(\n 'div',\n { className: 'gyxxwb-overlay' },\n createElement(WorkflowPanel, null),\n createElement(Toasts, null),\n )\n}\n\nexport function apply(ctx) {\n injectStyles()\n // \u63D2\u69FD\u987B\u7B49\u5BBF\u4E3B\u58F0\u660E\u540E\u624D\u80FD\u6CE8\u518C\uFF1Aslots.inject \u4F1A\u7B49\u5F85\u5EA7\u4F4D\u58F0\u660E\uFF08\u987A\u5E8F\u5B89\u5168\uFF09\uFF0C\n // \u76F4\u63A5 register \u5728\u6211\u4EEC\u5148\u4E8E ui-sidebar/ui-layout \u52A0\u8F7D\u65F6\u4F1A\u62A5\u201Cslot is not declared\u201D\u3002\n ctx.effect(\n () =>\n ctx.slots.inject('sidebar.footer.action', () =>\n ctx.slots.register(\n { name: 'sidebar.footer.action', id: 'gyxx-workbench.footer' },\n WorkbenchFooterAction,\n ),\n ),\n 'gyxx-workbench: footer action',\n )\n ctx.effect(\n () =>\n ctx.slots.inject('shell.overlay', () =>\n ctx.slots.register(\n { name: 'shell.overlay', id: 'gyxx-workbench.overlay' },\n WorkbenchOverlay,\n ),\n ),\n 'gyxx-workbench: overlay',\n )\n}\n", "/** \u5DE5\u4F5C\u6D41\u9762\u677F\uFF1A\u4FA7\u8FB9\u62BD\u5C49\uFF0C\u5217\u51FA\u5168\u90E8\u5DE5\u4F5C\u6D41\uFF0C\u9009\u4E2D\u540E\u53EF\u63D0\u95EE / \u8BCA\u65AD / \u4FEE\u590D / \u542F\u505C\u3002 */\n\nimport { useSyncExternalStore, useEffect, useCallback } from 'react'\nimport { getState, subscribe, update, pushToast } from './store.js'\nimport { api } from './api.js'\n\nfunction useWorkbench() {\n return useSyncExternalStore(subscribe, getState)\n}\n\nconst STATUS_LABELS = {\n success: '\u6210\u529F',\n failed: '\u5931\u8D25',\n running: '\u8FD0\u884C\u4E2D',\n cancelled: '\u5DF2\u505C\u6B62',\n none: '\u672A\u8FD0\u884C',\n}\n\nfunction statusOf(workflow) {\n if (workflow.active_run) return 'running'\n return workflow.last_run?.status ?? 'none'\n}\n\nfunction formatTime(value) {\n if (!value) return '\u2014'\n const date = new Date(value)\n if (Number.isNaN(date.getTime())) return String(value)\n const pad = (n) => String(n).padStart(2, '0')\n return `${pad(date.getMonth() + 1)}-${pad(date.getDate())} ${pad(date.getHours())}:${pad(date.getMinutes())}`\n}\n\nfunction formatDuration(seconds) {\n if (seconds == null) return ''\n if (seconds < 60) return `${seconds}s`\n return `${Math.floor(seconds / 60)}m${seconds % 60 ? `${seconds % 60}s` : ''}`\n}\n\nexport async function refreshWorkflows() {\n update({ loading: true, error: null })\n try {\n const payload = await api.listWorkflows()\n update({\n loading: false,\n workflows: payload.workflows ?? [],\n summary: payload.summary ?? null,\n generatedAt: payload.generated_at ?? null,\n })\n } catch (error) {\n update({ loading: false, error: error.message })\n }\n}\n\nasync function selectWorkflow(workflowId) {\n update({ selectedId: workflowId, detLine truncated
"mappings": ";;;;;;;;;;;;;;;;;;;;;;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAMA,IAAAA,gBAAoD;;;ACJpD,mBAA6D;;;ACG7D,IAAM,YAAY,oBAAI,IAAI;AAE1B,IAAI,QAAQ;AAAA,EACV,MAAM;AAAA,EACN,SAAS;AAAA,EACT,OAAO;AAAA,EACP,aAAa;AAAA,EACb,SAAS;AAAA,EACT,WAAW,CAAC;AAAA,EACZ,YAAY;AAAA,EACZ,QAAQ;AAAA,EACR,eAAe;AAAA,EACf,MAAM,CAAC;AAAA,EACP,WAAW;AAAA,EACX,kBAAkB;AAAA,EAClB,UAAU;AAAA,EACV,cAAc,oBAAoB;AAAA,EAClC,QAAQ,CAAC;AAAA,EACT,cAAc,CAAC;AAAA,EACf,QAAQ,CAAC;AACX;AAEO,SAAS,sBAAsB;AACpC,QAAM,MAAM,oBAAI,KAAK;AACrB,MAAI,QAAQ,IAAI,QAAQ,IAAI,CAAC;AAC7B,QAAM,MAAM,CAAC,UAAU,OAAO,KAAK,EAAE,SAAS,GAAG,GAAG;AACpD,SAAO,GAAG,IAAI,YAAY,CAAC,IAAI,IAAI,IAAI,SAAS,IAAI,CAAC,CAAC,IAAI,IAAI,IAAI,QAAQ,CAAC,CAAC;AAC9E;AAEO,SAAS,WAAW;AACzB,SAAO;AACT;AAEO,SAAS,OAAO,OAAO;AAC5B,UAAQ,EAAE,GAAG,OAAO,GAAG,MAAM;AAC7B,aAAW,YAAY,UAAW,UAAS;AAC7C;AAEO,SAAS,UAAU,UAAU;AAClC,YAAU,IAAI,QAAQ;AACtB,SAAO,MAAM,UAAU,OAAO,QAAQ;AACxC;AAEO,SAAS,UAAU,OAAO;AAC/B,QAAM,KAAK,KAAK,KAAK,IAAI,EAAE,SAAS,EAAE,CAAC,IAAI,KAAK,OAAO,EAAE,SAAS,EAAE,EAAE,MAAM,GAAG,CAAC,CAAC;AACjF,SAAO,EAAE,QAAQ,CAAC,GAAG,MAAM,QAAQ,EAAE,IAAI,GAAG,MAAM,CAAC,EAAE,CAAC;AACtD,SAAO;AACT;AAEO,SAAS,aAAa,IAAI;AAC/B,SAAO,EAAE,QAAQ,MAAM,OAAO,OAAO,CAAC,UAAU,MAAM,OAAO,EAAE,EAAE,CAAC;AACpE;;;ACtDO,SAAS,aAAa;AAC3B,MAAI,OAAO,WAAW,aAAa;AACjC,QAAI,OAAO,mBAAoB,QAAO,OAAO;AAC7C,QAAI;AACF,YAAM,QAAQ,OAAO,cAAc,QAAQ,gBAAgB;AAC3D,UAAI,MAAO,QAAO;AAAA,IACpB,QAAQ;AAAA,IAER;AAAA,EACF;AACA,SAAO;AACT;AAEA,eAAe,QAAQ,QAAQ,MAAM,MAAM;AACzC,QAAM,WAAW,MAAM,MAAM,GAAG,WAAW,CAAC,GAAG,IAAI,IAAI;AAAA,IACrD;AAAA,IACA,SAAS;AAAA,MACP,QAAQ;AAAA,MACR,GAAI,SAAS,SACT,EAAE,gBAAgB,oBAAoB,oBAAoB,IAAI,IAC9D,CAAC;AAAA,IACP;AAAA,IACA,MAAM,SAAS,SAAY,SAAY,KAAK,UAAU,IAAI;AAAA,EAC5D,CAAC;AACD,MAAI;AACJ,MAAI;AACF,cAAU,MAAM,SAAS,KAAK;AAAA,EAChC,QAAQ;AACN,cAAU,EAAE,IAAI,OAAO,OAAO,gFAAyB,SAAS,MAAM,SAAI;AAAA,EAC5E;AACA,MAAI,CAAC,SAAS,IAAI;AAChB,UAAM,IAAI,MAAM,SAAS,SAAS,kDAAe,SAAS,MAAM,QAAG;AAAA,EACrE;AACA,SAAO;AACT;AAEO,IAAM,MAAM;AAAA,EACjB,eAAe,MAAM,QAAQ,OAAO,mBAAmB;AAAA,EACvD,gBAAgB,CAAC,eACf,QAAQ,OAAO,qBAAqB,mBAAmB,UAAU,CAAC,EAAE;AAAA,EACtE,cAAc,CAAC,YAAY,QAAQ,MACjC,QAAQ,OAAO,qBAAqB,mBAAmB,UAAU,CAAC,eAAe,KAAK,EAAE;AAAA,EAC1F,WAAW,CAAC,YAAY,UACtB;AAAA,IACE;AAAA,IACA,qBAAqB,mBAAmB,UAAU,CAAC,aAAa,QAAQ,WAAW,mBAAmB,KAAK,CAAC,KAAK,EAAE;AAAA,EACrH;AAAA,EACF,QAAQ,MAAM,QAAQ,OAAO,gBAAgB;AAAA,EAC7C,eAAe,CAAC,OAAO,QAAQ,QAAQ,uBAAuB,EAAE,GAAG,CAAC;AAAA,EACpE,SAAS,CAAC,YAAY,QAAQ,QAAQ,mBAAmB,OAAO;AAAA,EAChE,QAAQ,CAAC,YAAY,QAAQ,QAAQ,kBAAkB,OAAO;AAAA,EAC9D,KAAK,CAAC,YAAY,QAAQ,QAAQ,eAAe,OAAO;AAC1D;;;AFwFM;AAxIN,SAAS,eAAe;AACtB,aAAO,mCAAqB,WAAW,QAAQ;AACjD;AAEA,IAAM,gBAAgB;AAAA,EACpB,SAAS;AAAA,EACT,QAAQ;AAAA,EACR,SAAS;AAAA,EACT,WAAW;AAAA,EACX,MAAM;AACR;AAEA,SAAS,SAAS,UAAU;AAC1B,MAAI,SAAS,WAAY,QAAO;AAChC,SAAO,SAAS,UAAU,UAAU;AACtC;AAEA,SAAS,WAAW,OAAO;AACzB,MAAI,CAAC,MAAO,QAAO;AACnB,QAAM,OAAO,IAAI,KAAK,KAAK;AAC3B,MAAI,OAAO,MAAM,KAAK,QAAQ,CAAC,EAAG,QAAO,OAAO,KAAK;AACrD,QAAM,MAAM,CAAC,MAAM,OAAO,CAAC,EAAE,SAAS,GAAG,GAAG;AAC5C,SAAO,GAAG,IAAI,KAAK,SAAS,IAAI,CAAC,CAAC,IAAI,IAAI,KAAK,QAAQ,CAAC,CAAC,IAAI,IAAI,KAAK,SAAS,CAAC,CAAC,IAAI,IAAI,KAAK,WAAW,CAAC,CAAC;AAC7G;AAEA,SAAS,eAAe,SAAS;AAC/B,MAAI,WAAW,KAAM,QAAO;AAC5B,MAAI,UAAU,GAAI,QAAO,GAAG,OAAO;AACnC,SAAO,GAAG,KAAK,MAAM,UAAU,EAAE,CAAC,IAAI,UAAU,KAAK,GAAG,UAAU,EAAE,MAAM,EAAE;AAC9E;AAEA,eAAsB,mBAAmB;AACvC,SAAO,EAAE,SAAS,MAAM,OAAO,KAAK,CAAC;AACrC,MAAI;AACF,UAAM,UAAU,MAAM,IAAI,cAAc;AACxC,WAAO;AAAA,MACL,SAAS;AAAA,MACT,WAAW,QAAQ,aAAa,CAAC;AAAA,MACjC,SAAS,QAAQ,WAAW;AAAA,MAC5B,aAAa,QAAQ,gBAAgB;AAAA,IACvC,CAAC;AAAA,EACH,SAAS,OAAO;AACd,WAAO,EAAE,SAAS,OAAO,OAAO,MAAM,QAAQ,CAAC;AAAA,EACjD;AACF;AAEA,eAAe,eAAe,YAAY;AACxC,SAAO,EAAE,YAAY,YAAY,eAAe,MAAM,WAAW,MAAM,MAAM,CAAC,EAAE,CAAC;AACjF,MAAI;AACF,UAAM,CAAC,QAAQ,IAAI,IAAI,MAAM,QAAQ,IAAI;AAAA,MACvC,IAAI,eAAe,UAAU,EAAE,MAAM,MAAM,IAAI;AAAA,MAC/C,IAAI,aAAa,YAAY,CAAC,EAAE,MAAM,OAAO,EAAE,MAAM,CAAC,EAAE,EAAE;AAAA,IAC5D,CAAC;AACD,WAAO,EAAE,eAAe,OAAO,QAAQ,MAAM,KAAK,QAAQ,CAAC,EAAE,CAAC;AAAA,EAChE,SAAS,OAAO;AACd,WAAO,EAAE,eAAe,OAAO,OAAO,MAAM,QAAQ,CAAC;AAAA,EACvD;AACF;AAEA,eAAe,cAAc,YAAY,OAAO;AAC9C,SAAO,EAAE,kBAAkB,KAAK,CAAC;AACjC,MAAI;AACF,UAAM,UAAU,MAAM,IAAI,UAAU,YAAY,KAAK;AACrD,WAAO,EAAE,kBAAkB,OAAO,WAAW,QAAQ,CAAC;AAAA,EACxD,SAAS,OAAO;AACd,WAAO,EAAE,kBAAkB,MAAM,CAAC;AAClC,cAAU,EAAE,MAAM,SAAS,MAAM,yDAAY,MAAM,OAAO,GAAG,CAAC;AAAA,EAChE;AACF;AAEA,eAAe,MAAM,QAAQ;AAC3B,QAAM,IAAI,SAAS;AACnB,MAAI,CAAC,EAAE,WAAY;AACnB,QAAM,WAAW,EAAE,SAAS,KAAK;AACjC,MAAI;AACF,UAAM,UAAU,MAAM,IAAI,IAAI;AAAA,MAC5B,aAAa,EAAE;AAAA,MACf;AAAA,MACA;AAAA,IACF,CAAC;AACD,UAAM,QAAQ,EAAE,KAAK,gBAAM,UAAU,gBAAM,QAAQ,eAAK,EAAE,MAAM,KAAK;AACrE,cAAU;AAAA,MACR,MAAM;AAAA,MACN,MAAM,qBAAM,KAAK,qBAAM,OAAO,QAAQ,UAAU,EAAE,MAAM,GAAG,EAAE,CAAC;AAAA,IAChE,CAAC;AACD,WAAO,EAAE,UAAU,GAAG,CAAC;AAAA,EACzB,SAAS,OAAO;AACd,cAAU,EAAE,MAAM,SAAS,MAAM,6CAAU,MAAM,OAAO,GAAG,CAAC;AAAA,EAC9D;AACF;AAEA,eAAe,UAAU,SAAS;AAChC,QAAM,IAAI,SAAS;AACnB,MAAI,CAAC,EAAE,WAAY;AACnB,MAAI,SAAS;AACX,UAAM,KAAK,OAAO;AAAA,MAChB,4BAAQ,EAAE,UAAU,kCAAS,EAAE,YAAY;AAAA,IAC7C;AACA,QAAI,CAAC,GAAI;AAAA,EACX;AACA,MAAI;AACF,UAAM,IAAI,QAAQ;AAAA,MAChB,aAAa,EAAE;AAAA,MACf,eAAe,EAAE;AAAA,MACjB;AAAA,MACA,WAAW;AAAA,IACb,CAAC;AACD,cALine truncated
"names": ["import_react", "import_react", "import_jsx_runtime", "useWorkbench", "useWorkbench"]
}
-481
View File
@@ -1,481 +0,0 @@
{
"name": "@gyxx/dsh-plugin-gyxx-workbench",
"version": "0.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@gyxx/dsh-plugin-gyxx-workbench",
"version": "0.1.0",
"devDependencies": {
"esbuild": "^0.24.0"
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.24.2.tgz",
"integrity": "sha512-thpVCb/rhxE/BnMLQ7GReQLLN8q9qbHmI55F4489/ByVg2aQaQ6kbcLb6FHkocZzQhxc4gx0sCk0tJkKBFzDhA==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.24.2.tgz",
"integrity": "sha512-tmwl4hJkCfNHwFB3nBa8z1Uy3ypZpxqxfTQOcHX+xRByyYgunVbZ9MzUUfb0RxaHIMnbHagwAxuTL+tnNM+1/Q==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.24.2.tgz",
"integrity": "sha512-cNLgeqCqV8WxfcTIOeL4OAtSmL8JjcN6m09XIgro1Wi7cF4t/THaWEa7eL5CMoMBdjoHOTh/vwTO/o2TRXIyzg==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.24.2.tgz",
"integrity": "sha512-B6Q0YQDqMx9D7rvIcsXfmJfvUYLoP722bgfBlO5cGvNVb5V/+Y7nhBE3mHV9OpxBf4eAS2S68KZztiPaWq4XYw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.24.2.tgz",
"integrity": "sha512-kj3AnYWc+CekmZnS5IPu9D+HWtUI49hbnyqk0FLEJDbzCIQt7hg7ucF1SQAilhtYpIujfaHr6O0UHlzzSPdOeA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.24.2.tgz",
"integrity": "sha512-WeSrmwwHaPkNR5H3yYfowhZcbriGqooyu3zI/3GGpF8AyUdsrrP0X6KumITGA9WOyiJavnGZUwPGvxvwfWPHIA==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.24.2.tgz",
"integrity": "sha512-UN8HXjtJ0k/Mj6a9+5u6+2eZ2ERD7Edt1Q9IZiB5UZAIdPnVKDoG7mdTVGhHJIeEml60JteamR3qhsr1r8gXvg==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.24.2.tgz",
"integrity": "sha512-TvW7wE/89PYW+IevEJXZ5sF6gJRDY/14hyIGFXdIucxCsbRmLUcjseQu1SyTko+2idmCw94TgyaEZi9HUSOe3Q==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.24.2.tgz",
"integrity": "sha512-n0WRM/gWIdU29J57hJyUdIsk0WarGd6To0s+Y+LwvlC55wt+GT/OgkwoXCXvIue1i1sSNWblHEig00GBWiJgfA==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.24.2.tgz",
"integrity": "sha512-7HnAD6074BW43YvvUmE/35Id9/NB7BeX5EoNkK9obndmZBUk8xmJJeU7DwmUeN7tkysslb2eSl6CTrYz6oEMQg==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.24.2.tgz",
"integrity": "sha512-sfv0tGPQhcZOgTKO3oBE9xpHuUqguHvSo4jl+wjnKwFpapx+vUDcawbwPNuBIAYdRAvIDBfZVvXprIj3HA+Ugw==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.24.2.tgz",
"integrity": "sha512-CN9AZr8kEndGooS35ntToZLTQLHEjtVB5n7dl8ZcTZMonJ7CCfStrYhrzF97eAecqVbVJ7APOEe18RPI4KLhwQ==",
"cpu": [
"loong64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.24.2.tgz",
"integrity": "sha512-iMkk7qr/wl3exJATwkISxI7kTcmHKE+BlymIAbHO8xanq/TjHaaVThFF6ipWzPHryoFsesNQJPE/3wFJw4+huw==",
"cpu": [
"mips64el"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.24.2.tgz",
"integrity": "sha512-shsVrgCZ57Vr2L8mm39kO5PPIb+843FStGt7sGGoqiiWYconSxwTiuswC1VJZLCjNiMLAMh34jg4VSEQb+iEbw==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.24.2.tgz",
"integrity": "sha512-4eSFWnU9Hhd68fW16GD0TINewo1L6dRrB+oLNNbYyMUAeOD2yCK5KXGK1GH4qD/kT+bTEXjsyTCiJGHPZ3eM9Q==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.24.2.tgz",
"integrity": "sha512-S0Bh0A53b0YHL2XEXC20bHLuGMOhFDO6GN4b3YjRLK//Ep3ql3erpNcPlEFed93hsQAjAQDNsvcK+hV90FubSw==",
"cpu": [
"s390x"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.24.2.tgz",
"integrity": "sha512-8Qi4nQcCTbLnK9WoMjdC9NiTG6/E38RNICU6sUNqK0QFxCYgoARqVqxdFmWkdonVsvGqWhmm7MO0jyTqLqwj0Q==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.24.2.tgz",
"integrity": "sha512-wuLK/VztRRpMt9zyHSazyCVdCXlpHkKm34WUyinD2lzK07FAHTq0KQvZZlXikNWkDGoT6x3TD51jKQ7gMVpopw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.24.2.tgz",
"integrity": "sha512-VefFaQUc4FMmJuAxmIHgUmfNiLXY438XrL4GDNV1Y1H/RW3qow68xTwjZKfj/+Plp9NANmzbH5R40Meudu8mmw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.24.2.tgz",
"integrity": "sha512-YQbi46SBct6iKnszhSvdluqDmxCJA+Pu280Av9WICNwQmMxV7nLRHZfjQzwbPs3jeWnuAhE9Jy0NrnJ12Oz+0A==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.24.2.tgz",
"integrity": "sha512-+iDS6zpNM6EnJyWv0bMGLWSWeXGN/HTaF/LXHXHwejGsVi+ooqDfMCCTerNFxEkM3wYVcExkeGXNqshc9iMaOA==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.24.2.tgz",
"integrity": "sha512-hTdsW27jcktEvpwNHJU4ZwWFGkz2zRJUz8pvddmXPtXDzVKTTINmlmga3ZzwcuMpUvLw7JkLy9QLKyGpD2Yxig==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.24.2.tgz",
"integrity": "sha512-LihEQ2BBKVFLOC9ZItT9iFprsE9tqjDjnbulhHoFxYQtQfai7qfluVODIYxt1PgdoyQkz23+01rzwNwYfutxUQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.24.2.tgz",
"integrity": "sha512-q+iGUwfs8tncmFC9pcnD5IvRHAzmbwQ3GPS5/ceCyHdjXubwQWI12MKWSNSMYLJMq23/IUCvJMS76PDqXe1fxA==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.24.2.tgz",
"integrity": "sha512-7VTgWzgMGvup6aSqDPLiW5zHaxYJGTO4OokMjIlrCtf+VpEL+cXKtCvg723iguPYI5oaUNdS+/V7OU2gvXVWEg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/esbuild": {
"version": "0.24.2",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.24.2.tgz",
"integrity": "sha512-+9egpBW8I3CD5XPe0n6BfT5fxLzxrlDzqydF3aviG+9ni1lDC/OvMHcxqEFV0+LANZG5R1bFMWfUrjVsdwxJvA==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"bin": {
"esbuild": "bin/esbuild"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.24.2",
"@esbuild/android-arm": "0.24.2",
"@esbuild/android-arm64": "0.24.2",
"@esbuild/android-x64": "0.24.2",
"@esbuild/darwin-arm64": "0.24.2",
"@esbuild/darwin-x64": "0.24.2",
"@esbuild/freebsd-arm64": "0.24.2",
"@esbuild/freebsd-x64": "0.24.2",
"@esbuild/linux-arm": "0.24.2",
"@esbuild/linux-arm64": "0.24.2",
"@esbuild/linux-ia32": "0.24.2",
"@esbuild/linux-loong64": "0.24.2",
"@esbuild/linux-mips64el": "0.24.2",
"@esbuild/linux-ppc64": "0.24.2",
"@esbuild/linux-riscv64": "0.24.2",
"@esbuild/linux-s390x": "0.24.2",
"@esbuild/linux-x64": "0.24.2",
"@esbuild/netbsd-arm64": "0.24.2",
"@esbuild/netbsd-x64": "0.24.2",
"@esbuild/openbsd-arm64": "0.24.2",
"@esbuild/openbsd-x64": "0.24.2",
"@esbuild/sunos-x64": "0.24.2",
"@esbuild/win32-arm64": "0.24.2",
"@esbuild/win32-ia32": "0.24.2",
"@esbuild/win32-x64": "0.24.2"
}
}
}
}
-30
View File
@@ -1,30 +0,0 @@
{
"name": "@gyxx/dsh-plugin-gyxx-workbench",
"version": "0.1.0",
"private": true,
"description": "GYXX 智能工作台:把 gyxx-flow 工作流暴露为 deepseek-harness 智能体工具与侧边栏面板",
"type": "module",
"main": "gyxx-workbench.mjs",
"exports": {
".": "./gyxx-workbench.mjs",
"./client": "./lib/client.js",
"./package.json": "./package.json"
},
"dsh": {
"client": {
"platform": "web",
"inject": [
"@deepseek-ai/dsh-client-ui-renderer",
"@deepseek-ai/dsh-client-ui-layout",
"@deepseek-ai/dsh-client-locale"
]
}
},
"scripts": {
"build": "node scripts/build-client.mjs",
"test": "node --test \"tests/**/*.test.mjs\""
},
"devDependencies": {
"esbuild": "^0.24.0"
}
}
-51
View File
@@ -1,51 +0,0 @@
/**
* 构建客户端包:esbuild(CJS / browser / 平台模块外置)+ dsh 模块加载器包装。
* 包装协议与 dsh 官方 tsdown 产物一致:
* window.__ModuleLoader__.load({ id, factory: (require) => { ... return module.exports } })
*/
import { build } from 'esbuild'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
const root = join(dirname(fileURLToPath(import.meta.url)), '..')
const PKG_ID = '@gyxx/dsh-plugin-gyxx-workbench'
// dsh 平台共享模块表(packages/client/web/src/platform.ts 的 PLATFORM_MODULES)
const PLATFORM_EXTERNALS = [
'react',
'react/jsx-runtime',
'react-dom',
'react-dom/client',
'@deepseek-ai/cordis',
'@deepseek-ai/dsh-client-store',
'@deepseek-ai/dsh-client-ui-slots',
'@deepseek-ai/dsh-client-ui-primitives',
]
await build({
absWorkingDir: root,
entryPoints: ['client/src/index.jsx'],
bundle: true,
format: 'cjs',
platform: 'browser',
target: 'es2020',
outfile: 'lib/client.js',
external: PLATFORM_EXTERNALS,
loader: { '.css': 'text' },
jsx: 'automatic',
sourcemap: true,
minify: false,
define: {
'process.env.NODE_ENV': JSON.stringify(process.env.NODE_ENV ?? 'production'),
},
banner: {
// dsh 浏览器加载器只给 factory 传 require(返回值即模块导出),
// CJS 的 module/exports 垫片必须由包体自带(与官方 tsdown 产物一致)。
js: `window.__ModuleLoader__.load({ id: ${JSON.stringify(PKG_ID)}, factory: (require) => {\nvar module = { exports: {} };\nvar exports = module.exports;`,
},
footer: { js: 'return module.exports; } });' },
logLevel: 'info',
})
console.log('built lib/client.js')
-164
View File
@@ -1,164 +0,0 @@
/**
* gyxx-workbench 宿主插件冒烟测试:
* - apply() 能注册 7 个工具与系统提示词
* - 工具在控制台离线时返回结构化错误(不抛出)
* - 桥接服务能代理控制台 API 并为提问创建会话
* 运行:node --test tests/
*/
import test from 'node:test'
import assert from 'node:assert/strict'
import http from 'node:http'
import * as plugin from '../gyxx-workbench.mjs'
function makeCtx() {
const registered = { tools: [], sections: [], effects: [], emissions: [] }
const agents = {
calls: [],
async create(options) {
agents.calls.push(options)
return {
agent: {
followups: [],
followup(message) {
this.followups.push(message)
agents.calls[agents.calls.length - 1].followup = message
},
},
dispose() {},
}
},
}
const ctx = {
tools: { register: (def) => registered.tools.push(def) },
systemPrompt: { section: (section) => registered.sections.push(section) },
effect(fn) {
registered.effects.push(fn)
return undefined
},
emit: (event, payload) => registered.emissions.push([event, payload]),
get: (key) => (key === 'agents' ? agents : undefined),
logger: { info() {} },
}
return { ctx, registered, agents }
}
test('apply 注册 7 个工作流工具与系统提示词', () => {
const { ctx, registered } = makeCtx()
plugin.apply(ctx, { bridgePort: 0, pollIntervalMs: 60_000 })
const names = registered.tools.map((tool) => tool.name).sort()
assert.deepEqual(names, [
'gyxx_schedule_update',
'gyxx_workflow_cancel',
'gyxx_workflow_detail',
'gyxx_workflow_diagnose',
'gyxx_workflow_list',
'gyxx_workflow_runs',
'gyxx_workflow_trigger',
])
assert.equal(registered.sections.length, 1)
assert.equal(registered.sections[0].name, 'gyxx-workbench')
assert.match(registered.sections[0].text, /安全契约/)
// 监控器 + 桥接服务两个 effect
assert.equal(registered.effects.length, 2)
})
test('工具在控制台离线时返回结构化错误而非抛出', async () => {
const { ctx, registered } = makeCtx()
plugin.apply(ctx, {
consoleBaseUrl: 'http://127.0.0.1:1',
bridgePort: 0,
pollIntervalMs: 60_000,
})
const listTool = registered.tools.find((tool) => tool.name === 'gyxx_workflow_list')
const result = await listTool.execute({})
assert.equal(result.ok, false)
assert.match(result.error, /无法连接 gyxx 控制台/)
})
test('正式执行缺少 confirmed 时工具直接拒绝', async () => {
const { ctx, registered } = makeCtx()
plugin.apply(ctx, { consoleBaseUrl: 'http://127.0.0.1:1', bridgePort: 0 })
const trigger = registered.tools.find((tool) => tool.name === 'gyxx_workflow_trigger')
const result = await trigger.execute({
workflow_id: 'content.metrics.daily',
business_date: '2026-08-01',
execute: true,
})
assert.equal(result.ok, false)
assert.match(result.error, /确认/)
})
test('桥接服务代理控制台 API 并通过 /bridge/ask 创建会话', async (t) => {
// 1) 模拟 gyxx 控制台
const consoleServer = http.createServer((req, res) => {
if (req.url === '/api/overview') {
res.writeHead(200, { 'content-type': 'application/json' })
res.end(
JSON.stringify({
workflows: [
{
id: 'content.metrics.daily',
name: '内容指标日报',
module: 'content_marketing',
trigger: 'scheduled',
registered: true,
schedule: { enabled: true, kind: 'daily', at: '09:00' },
last_run: {
run_id: 'run-1',
business_date: '2026-08-01',
mode: 'execute',
shadow: false,
status: 'failed',
started_at: '2026-08-01T01:00:00+00:00',
ended_at: '2026-08-01T01:05:00+00:00',
error: 'step failed',
step_counts: { failed: 1 },
},
active_run: null,
},
],
summary: { total: 1 },
}),
)
return
}
if (req.url === '/api/workflows/content.metrics.daily') {
res.writeHead(200, { 'content-type': 'application/json' })
res.end(JSON.stringify({ workflow: { id: 'content.metrics.daily', name: '内容指标日报' } }))
return
}
res.writeHead(404, { 'content-type': 'application/json' })
res.end(JSON.stringify({ error: '页面或接口不存在' }))
})
await new Promise((resolve) => consoleServer.listen(0, '127.0.0.1', resolve))
t.after(() => consoleServer.close())
const consolePort = consoleServer.address().port
// 2) 以 mock ctx 加载插件并手动启动桥接 effect
const { ctx, registered, agents } = makeCtx()
plugin.apply(ctx, {
consoleBaseUrl: `http://127.0.0.1:${consolePort}`,
bridgePort: 0,
pollIntervalMs: 600_000,
})
const bridgeEffect = registered.effects[1]
const dispose = await bridgeEffect()
t.after(() => dispose())
// 桥接端口由 effect 内部 listen 决定,通过日志无法读取——改为直接请求
// 控制台代理行为经工具路径验证;这里验证 ask 会话创建逻辑。
const listTool = registered.tools.find((tool) => tool.name === 'gyxx_workflow_list')
const list = await listTool.execute({})
assert.equal(list.ok, true)
assert.equal(list.workflows.length, 1)
assert.equal(list.workflows[0].last_run.status, 'failed')
assert.equal(list.workflows[0].module_label, '内容营销')
// diagnose:自动选择最近失败运行(控制台无该路由时应返回结构化错误)
const diagnose = registered.tools.find((tool) => tool.name === 'gyxx_workflow_diagnose')
const missing = await diagnose.execute({ workflow_id: 'content.metrics.daily' })
assert.equal(missing.ok, false)
assert.match(missing.error, /页面或接口不存在|HTTP 404/)
})