wangyunlong/gyxx-flow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: expand module workflows, dynamic config, notifications and console API

Browse Source
This commit is contained in:
wangyunlong
2026-09-04 09:49:37 +08:00
parent 8df5266abb
commit b124d757b0
309 changed files with 89358 additions and 6232 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+35 -22
View File
@@ -2,38 +2,51 @@
## Project Structure & Module Organization
GYXX Flow is a Python 3.12 `src`-layout package. Framework code lives in `src/gyxx_flow/`; business implementations belong directly in `src/gyxx_flow/modules/<module>/`. The former `runtime/` directories are compatibility namespaces only and must not receive new production code.
GYXX Flow is a Python 3.12 `src`-layout modular monolith: LangGraph workflow orchestration, a Python-resident scheduler, and per-run audit (journal, locks, effect ledger). Docs, commit messages, and config descriptions are largely Chinese — follow the existing language of each file.
All project and module tests live in `tests/`. Configuration lives in `config/`, runbooks in `docs/`, and service scripts in `deploy/`. Treat `var/` as generated state, not source; production deployments must set an external `GYXX_DATA_ROOT`.
- `src/gyxx_flow/core/`, `workflow/`, `adapters/` — shared framework: settings/`RunContext`/journal/locks (`core`), validated workflow models + `build_catalog_workflow` + `WorkflowEngine` (`workflow`), and the only sanctioned bridges to PostgreSQL/Feishu/Hermes/browser (`adapters`).
- `src/gyxx_flow/modules/<module>/` — business code for `content_marketing`, `product_commerce`, `shop_intelligence`, `supply_chain`. Modules depend only on shared contracts, never on another module's internals. The `runtime/` subpackages are legacy import-compatibility namespaces only; new production code goes directly in the module directory.
- `config/` — the validated configuration quartet: `workflows.json` (scheduled workflows only, exactly one schedule each), `schedules.json` (time rules), `commands.json` (manual-command whitelist), `runtime-bindings.json` (browser bindings + accounts).
- `tests/`, `docs/`, `deploy/` — tests, runbooks, deployment. `var/` is generated state (gitignored); production must set an external `GYXX_DATA_ROOT`.
## Build, Test, and Development Commands
Use `uv` from the repository root:
Everything runs through `uv` from the repository root:
```powershell
uv sync --python 3.12 --group dev # create/update the development environment
uv run pytest # run the project test suite
uv run pytest tests/test_cli.py # run a focused test file
uv run ruff check src tests # lint imports and Python errors
uv build # create distribution artifacts
uv run gyxx doctor --json # validate local configuration
uv run gyxx list # inspect registered workflows
uv sync --python 3.12 --group dev # create/update the dev environment
uv run pytest # full suite
uv run pytest tests/test_cli.py # single file
uv run ruff check src tests # lint (E4, E7, E9, F, I)
uv build # distribution artifacts
uv run gyxx doctor --json # environment/config preflight
uv run gyxx acceptance status --json # acceptance evidence
uv run gyxx sources status --source-root <module>=<dir> # upstream drift check
```
Workflow and command execution defaults to dry-run behavior. Add `--execute` only when real external effects are intended. Public manual commands must be declared in `config/commands.json`; never reintroduce recursive executable-script discovery.
Manual execution is dry-run by default; real external side effects require explicit `--execute`:
Production schedules run only through the Python scheduler service; do not add Windows Task Scheduler registration. Manual commands remain dry-run by default, while `gyxx schedule run` executes enabled jobs unless `--dry-run` is supplied. PostgreSQL uses the cloud service selected through runtime-only credentials, both Hermes roles remain loopback-only, Feishu keeps the existing integration behavior, credentials must not have non-empty source defaults, and every browser entry must retain its unique CDP/Profile/Cookie/storage-state binding.
```powershell
uv run gyxx list # scheduled workflows only
uv run gyxx run product.daily --date 2026-08-01 [--execute]
uv run gyxx backfill content.metrics.daily --from <d> --to <d> [--execute]
uv run gyxx scripts run <script_id> --date <d> [--execute]
uv run gyxx schedule run [--dry-run] [--once] # executes enabled jobs unless --dry-run
uv run gyxx accounts login|sync [account_id]
```
Each executable `workflow_id` compiles to a LangGraph `StateGraph`. Preserve the public workflow model, journal, lock, shadow, and effect-ledger contracts when changing graph execution. Check upstream drift with `uv run gyxx sources status`; automated source application is limited to untransformed, conflict-free entries and still requires `--execute`.
## Constraints (preserve these contracts)
## Coding Style & Naming Conventions
- `gyxx schedule run` is the only production scheduling path; never add Windows Task Scheduler entries or duplicate cron rules.
- Manual/backfill/maintenance commands must be declared in `config/commands.json`; never reintroduce recursive executable-script discovery.
- Each executable `workflow_id` compiles to a LangGraph `StateGraph`. Preserve the public contracts of the workflow model, `RunJournal`, `LockManager`, shadow comparison, and `EffectLedger` when changing graph execution.
- External systems: PostgreSQL via runtime-injected cloud DSN (`GYXX_POSTGRES_DSN`; no address/credential defaults in source, loopback rejected in cloud mode); Hermes loopback-only roles on `127.0.0.1:8642/8643` (key `GYXX_HERMES_API_KEY`); Feishu keeps its existing identity. Every browser binding keeps a unique CDP port plus isolated Profile/Cookie/storage-state; scripts sharing an account declare it under `accounts` in `runtime-bindings.json` — the vault `state/accounts/<id>/` is the single login authority (`gyxx accounts login/sync`) and its cookies merge read-only into member scripts while profiles stay isolated.
- 启动或重启工作流控制台（`gyxx console`）必须携带 `--env-file D:\product-collector-analyze-flow\.env`（与调度器进程一致），否则 `PG_HOST/PG_PORT/PG_DB/PG_USER/PG_PASSWORD` 等云端 PostgreSQL 凭据和账号配置不会注入，「商品经营日报采集」「聚水潭全店铺款式日报」等正式执行会报“缺少 PostgreSQL 运行时凭据”。参考启动脚本：`var/tmp/start-console.ps1`。
- Source sync (`gyxx sources status/apply`): manifest-driven three-way hash drift check; only untransformed, conflict-free one-sided changes may be auto-copied, and only with `--execute`. Credentials, cookies, logs, and collected data never enter sync manifests.
Follow existing Python conventions: four-space indentation, type annotations, concise docstrings, and imports ordered as standard library, third party, then local modules. Ruff enforces `E4`, `E7`, `E9`, `F`, and `I` rules. Use `snake_case` for modules, functions, variables, and test files; `PascalCase` for classes; and uppercase names for constants. Keep cross-module contracts in shared framework packages rather than importing another business module’s internals.
## Coding Style & Testing
## Testing Guidelines
Tests use pytest and follow `test_*.py` / `test_*` naming. Add focused unit tests for new logic and regression tests for fixed defects. Changes to workflows, schedules, manifests, runtime paths, or CLI behavior should include acceptance or boundary coverage. Avoid tests that require live credentials or mutate production services.
## Commit & Pull Request Guidelines
Use short Conventional Commit subjects such as `feat: consolidate legacy workflows`. Keep commits scoped. Pull requests should explain behavior, list verification, link issues, and call out configuration or migration effects. Never commit secrets, cookies, tokens, `.env` files, or generated `var/` contents.
- Ruff enforces `E4`, `E7`, `E9`, `F`, `I`. `pyproject.toml` excludes `src/gyxx_flow/modules/*/**` from lint (legacy baseline); CI additionally lints a focused list of critical module scripts — add newly critical module scripts there rather than removing the exclusion.
- Tests are pytest under `tests/` (`test_*.py`); never require live credentials or mutate production services. Changes to workflows, schedules, manifests, runtime paths, or CLI behavior need acceptance or boundary coverage.
- Commits use short Conventional Commit subjects (e.g. `feat: consolidate legacy workflows`); never commit secrets, cookies, `.env` files, or generated `var/` contents.
- Operational detail: `docs/architecture.md`, `docs/deployment.md`, `docs/runbook.md`, `docs/acceptance-report.md`.