feat: expand module workflows, dynamic config, notifications and console API
This commit is contained in:
@@ -0,0 +1,73 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project
|
||||
|
||||
GYXX Flow is a Python 3.12 modular-monolith business automation platform. It orchestrates 23 scheduled workflows across four business domains (content marketing, product commerce, shop intelligence, supply chain) with LangGraph, runs them through a resident Python scheduler, and audits every run. Documentation, commit messages, and config descriptions are largely in Chinese; follow the existing language of each file.
|
||||
|
||||
## Common Commands
|
||||
|
||||
Everything runs through `uv` from the repo root:
|
||||
|
||||
```powershell
|
||||
uv sync --python 3.12 --group dev # create/update dev environment
|
||||
uv run pytest # full test suite
|
||||
uv run pytest tests/test_cli.py # single test file
|
||||
uv run pytest tests/path/test_x.py::test_name -k pattern # focused tests
|
||||
uv run ruff check src tests # lint (E4, E7, E9, F, I)
|
||||
uv build # distribution artifacts
|
||||
```
|
||||
|
||||
CLI (entry point `gyxx_flow.cli:main`). Manual execution is **dry-run by default**; real external side effects require an explicit `--execute` flag:
|
||||
|
||||
```powershell
|
||||
uv run gyxx list # scheduled workflows only
|
||||
uv run gyxx run product.daily --date 2026-08-01 # dry-run one workflow
|
||||
uv run gyxx run product.daily --date 2026-08-01 --execute
|
||||
uv run gyxx backfill content.metrics.daily --from <d> --to <d>
|
||||
uv run gyxx scripts run <script_id> --date <d> # manual backfill/retry/maintenance
|
||||
uv run gyxx schedule run --dry-run --once # scheduler; omit --dry-run to execute
|
||||
uv run gyxx doctor --json # environment/config preflight
|
||||
uv run gyxx acceptance status --json
|
||||
uv run gyxx sources status --source-root <module>=<dir> # upstream source drift check
|
||||
```
|
||||
|
||||
Manual/backfill commands must be declared in `config/commands.json`; never reintroduce recursive executable-script discovery. `gyxx schedule run` is the only production scheduling path — do not add Windows Task Scheduler tasks or extra cron entries.
|
||||
|
||||
## Architecture
|
||||
|
||||
**Execution pipeline:** CLI / `scheduler_service.py` → `WorkflowCatalog` (parses `config/workflows.json`) → `WorkflowRegistry` + per-module factories → `build_catalog_workflow` (`workflow/factory.py`) compiles each workflow into a LangGraph `StateGraph` → `WorkflowEngine` (`workflow/engine.py`) executes steps with retries, resource locks, timeouts, and effect-ledger guards. Every run gets a stable `run_id` and records node status, logs, artifacts, locks, and external side effects in the `RunJournal` and `EffectLedger` (`ops/effects.py`). Preserve the public contracts of the workflow model, journal, locks, shadow comparison, and effect ledger when changing graph execution.
|
||||
|
||||
**Layer boundaries** (enforced by tests):
|
||||
|
||||
- `src/gyxx_flow/core/` — settings, `RunContext`, `DataLayout`, `LockManager`, `RunJournal`, artifacts, exit codes.
|
||||
- `src/gyxx_flow/workflow/` — `StepDefinition`/`WorkflowDefinition` (validated models), `ExecutableStep` protocol, graph compilation, engine.
|
||||
- `src/gyxx_flow/adapters/` — the only sanctioned bridge to infrastructure: PostgreSQL/Feishu/Hermes outboxes (`external.py`), browser profile/CDP management (`browser.py`), runtime service binding for child scripts (`integration.py`), module command resolution (`native.py`), acceptance/cookie policy (`acceptance_policy.py`).
|
||||
- `src/gyxx_flow/modules/<module>/` — the four business modules (`content_marketing`, `product_commerce`, `shop_intelligence`, `supply_chain`). Business modules may depend only on shared contracts above — **never on another module's internals**. New production code goes directly in the module directory; the `runtime/` subpackages are legacy import-compatibility namespaces only.
|
||||
- `src/gyxx_flow/ops/`, `source_sync/`, `security/`, `baseline/` — effect ledger/run index, manifest-driven source-drift tooling, secret scanning, engineering baselines.
|
||||
|
||||
**Configuration quartet** in `config/` (all are validated at startup):
|
||||
|
||||
- `workflows.json` — scheduled workflows only: stable id, module, explicit steps (entry script, args, `depends_on`, `run_after_failure`), failure policy. Each entry must have exactly one schedule.
|
||||
- `schedules.json` — time rules only (all `Asia/Shanghai`; a day plan may have multiple times). Scheduler (`scheduler_service.py`) owns business-date offsets, slot dedup, non-overlap, missed-trigger compensation, graceful stop.
|
||||
- `commands.json` — manual-execution whitelist (backfill, retry, mapping refresh, protected writes). `--date` renders a command's declared `{business_date}` defaults; these are never new scheduled workflows.
|
||||
- `runtime-bindings.json` — each browser script gets a unique CDP port plus isolated Profile/Cookie/storage-state; never share writable profiles. Scripts that use the same account may declare an `account` id (see the top-level `accounts` section): the account vault under `state/accounts/<id>/` is the single login authority, and its cookies are merged read-only into every member script's cookie files before each run (`gyxx accounts login/sync`). Member profiles stay isolated so parallel runs never contend.
|
||||
|
||||
**External systems** (all accessed through adapters):
|
||||
|
||||
- PostgreSQL: runtime-injected cloud DSN via `GYXX_POSTGRES_DSN` (mapped to `PG_*`/`DB_*`/`AUTOFLOW_PG_*`). No address/credentials defaults in source; loopback DB addresses are rejected in cloud mode.
|
||||
- Hermes: loopback-only roles — analyzer `http://127.0.0.1:8642/v1`, collector `http://127.0.0.1:8643/v1`; key via `GYXX_HERMES_API_KEY`. Non-loopback Hermes addresses are rejected. Pure-collection workflows run without AI; analysis/notification workflows need the local roles.
|
||||
- Feishu: keeps the existing lark-cli profile/app identity.
|
||||
- Steps declare `production_sink`, `official_notification`, and `replay_policy` (`guarded`/`idempotent`); the engine and effect ledger enforce these gates.
|
||||
|
||||
**Data layout:** all runtime paths derive from `GYXX_DATA_ROOT` (dev default `var/`, which is gitignored generated state — never commit it or write source into it; production must use an external root). Layers: `data/{raw,normalized,curated,exports,evidence}/<module>`, `state/` (scheduler, graph, locks, ledger, per-script browser state), `logs/`, `tmp/`. Raw data is append-only.
|
||||
|
||||
## Repo-Specific Rules
|
||||
|
||||
- **Ruff lint baseline:** `src/gyxx_flow/modules/*/**` is excluded via `extend-exclude` in `pyproject.toml` (migrated scripts keep their old baseline). CI additionally lints a focused list of critical module scripts (see `.gitlab-ci.yml`); when adding or substantially changing module scripts that must be linted, add them to that focused CI list.
|
||||
- **Adding a scheduled workflow:** implement an independently testable entry in `modules/<module>/`, declare steps in `workflows.json`, add exactly one schedule rule, register commands and browser bindings as needed, access infrastructure only through shared adapters, and add dry-run/boundary/business-date regression tests. Backfill/maintenance capabilities are registered as commands only — they become workflows only when they become independent recurring jobs. New modules must not require changes to other modules; new steps must not require changes to the scheduler or engine.
|
||||
- **Source sync** (`gyxx sources status/apply`): manifest-driven three-way hash comparison against upstream module source dirs; only untransformed, conflict-free one-sided changes may be auto-copied, and only with `--execute`. Cookies, profiles, credentials, logs, and collected data never enter sync manifests.
|
||||
- Tests live in `tests/` (pytest, `test_*.py`); never require live credentials or mutate production services. Changes to workflows, schedules, manifests, runtime paths, or CLI behavior need acceptance or boundary coverage.
|
||||
- Use Conventional Commit subjects (e.g. `feat: consolidate legacy workflows`). Never commit secrets, cookies, `.env` files, or `var/` contents.
|
||||
- See `docs/architecture.md`, `docs/deployment.md`, `docs/runbook.md`, and `docs/acceptance-report.md` for operational detail.
|
||||
Reference in New Issue
Block a user