feat(product-commerce): 款式主键统一 style_id 并下线评价采集
采集端与 data-hub 统一用 style_id (= dim_style.id) 关联款式,style_name 降级为显示冗余。
款式主键:
- 14 张采集表加 style_id BIGINT + style_match_method + *_style_id_idx 索引
- 一律逻辑外键:不建 FOREIGN KEY,早期 10 个 *_style_id_fk 在 schema.sql 里幂等 DROP
- loader 下发 dim_style.id;db 层 attach_style_ids / resolve_style_ids_by_product 兜底
- 新增 backfill_style_id.py 三段回填(款名 / 日报按商品 ID / 广告表按 spu_id)
- 新增视图 v_style_id_coverage、v_style_unmatched_product、v_style_unmatched_source
- 实库补录 199 个商品 ID 到 67 款 spus;日报 style_id 覆盖 49.1% -> 79.0%,
tm 广告 99.4% / jd 92.4% / dy 83.2%,其余 11 张 100%
评价采集下线:
- product_reviews 归档为 product_reviews_archived_20261101
- 差评改由 shop.{tmall,jd,dy}_negative_reviews.daily 采集,消费方读 v_negative_reviews
- 删除 import_product_reviews.py / orchestrate_review_collection.py 及相关命令与绑定
- dy_product_scraping 默认流程摘除评论采集,保留 --reviews-only
天猫广告:
- 迁移到独立表 tm_product_ad_daily_metrics,不再写 product_daily_metrics.raw_data
同批附带:
- 光合上传支持天猫新店 new_bag(仅黑曜石系列)
- 京东商智双月日历排除 -diff-month 补位格
- 万相主体搜索词覆盖 STYLE_SEARCH_KEYWORDS
- ERP 部分编码失败时用已成功编码销量兜底,避免整款归零
This commit is contained in:
1 parent
622a609800
commit
6f1c193f4b
50 files changed
+3301
-1150
No files matched your search
@@ -83,7 +83,7 @@ def test_default_registry_covers_every_executable_workflow_entry() -> None:
|
||||
assert scheduled_workflows == 40
|
||||
assert manual_workflows == 0
|
||||
assert len(workflow_entries) == 56
|
||||
assert len(catalog.scripts) == 62
|
||||
assert len(catalog.scripts) == 61
|
||||
assert workflow_entries < set(catalog.script_ids)
|
||||
assert {
|
||||
"content.mapping.rebuild",
|
||||
@@ -97,7 +97,6 @@ def test_default_registry_covers_every_executable_workflow_entry() -> None:
|
||||
"shop.douyin_after_sales_quality.collect",
|
||||
"product.backfill.run",
|
||||
"product.erp_all_shop_daily",
|
||||
"product.review.orchestrate",
|
||||
"product.video_upload.run",
|
||||
"product.jd_video_upload.run",
|
||||
"supply.workflow.run",
|
||||
@@ -147,10 +146,6 @@ def test_default_registry_covers_every_executable_workflow_entry() -> None:
|
||||
"--to",
|
||||
"{business_date}",
|
||||
)
|
||||
assert catalog.get("product.review.orchestrate").default_args == (
|
||||
"--target-date",
|
||||
"{business_date}",
|
||||
)
|
||||
assert catalog.get("product.import.tmall_ads").default_args == (
|
||||
"--date",
|
||||
"{business_date}",
|
||||
@@ -280,7 +275,7 @@ def test_cli_lists_and_dry_runs_registered_commands(tmp_path: Path) -> None:
|
||||
|
||||
assert main(["scripts", "list", "--json"], settings=settings, stdout=output) == 0
|
||||
rows = json.loads(output.getvalue())
|
||||
assert len(rows) == 62
|
||||
assert len(rows) == 61
|
||||
assert {row["module"] for row in rows} == {
|
||||
"content_marketing",
|
||||
"product_commerce",
|
||||
|
||||
Reference in new issue
Block a user