feat(product-commerce): 款式主键统一 style_id 并下线评价采集

采集端与 data-hub 统一用 style_id (= dim_style.id) 关联款式,style_name 降级为显示冗余。

款式主键:
- 14 张采集表加 style_id BIGINT + style_match_method + *_style_id_idx 索引
- 一律逻辑外键:不建 FOREIGN KEY,早期 10 个 *_style_id_fk 在 schema.sql 里幂等 DROP
- loader 下发 dim_style.id;db 层 attach_style_ids / resolve_style_ids_by_product 兜底
- 新增 backfill_style_id.py 三段回填(款名 / 日报按商品 ID / 广告表按 spu_id)
- 新增视图 v_style_id_coverage、v_style_unmatched_product、v_style_unmatched_source
- 实库补录 199 个商品 ID 到 67 款 spus;日报 style_id 覆盖 49.1% -> 79.0%,
  tm 广告 99.4% / jd 92.4% / dy 83.2%,其余 11 张 100%

评价采集下线:
- product_reviews 归档为 product_reviews_archived_20261101
- 差评改由 shop.{tmall,jd,dy}_negative_reviews.daily 采集,消费方读 v_negative_reviews
- 删除 import_product_reviews.py / orchestrate_review_collection.py 及相关命令与绑定
- dy_product_scraping 默认流程摘除评论采集,保留 --reviews-only

天猫广告:
- 迁移到独立表 tm_product_ad_daily_metrics,不再写 product_daily_metrics.raw_data

同批附带:
- 光合上传支持天猫新店 new_bag(仅黑曜石系列)
- 京东商智双月日历排除 -diff-month 补位格
- 万相主体搜索词覆盖 STYLE_SEARCH_KEYWORDS
- ERP 部分编码失败时用已成功编码销量兜底,避免整款归零
This commit is contained in:
wangyunlong committed 2026-11-01 16:43:27 +08:00
1 parent 622a609800
commit 6f1c193f4b
50 files changed
+3301 -1150

No files matched your search

+11 -3
View File
@@ -102,10 +102,10 @@ def test_project_registry_covers_every_script_with_unique_stable_port() -> None:
data_root=project_root / "var",
)
assert len(scripts.command_ids) == 62
assert len(first.command_ids) == 157
assert len(scripts.command_ids) == 61
assert len(first.command_ids) == 155
bindings = [first.binding_for(command_id) for command_id in first.command_ids]
assert len({binding.cdp_port for binding in bindings}) == 157
assert len({binding.cdp_port for binding in bindings}) == 155
assert all(22000 <= binding.cdp_port <= 22999 for binding in bindings)
assert all(binding.cdp_url.startswith("http://127.0.0.1:") for binding in bindings)
assert bindings == [second.binding_for(item.script_id) for item in bindings]
@@ -131,6 +131,8 @@ def test_project_registry_covers_every_script_with_unique_stable_port() -> None:
"GUANGHE_PASSWORD",
"GUANGHE_LUGGAGE_USERNAME",
"GUANGHE_LUGGAGE_PASSWORD",
"GUANGHE_NEW_BAG_USERNAME",
"GUANGHE_NEW_BAG_PASSWORD",
)
video_environment = first.environment_for("product.video_upload.run", {})
assert video_environment["GUANGHE_USER_DATA_DIR"] == str(
@@ -142,6 +144,12 @@ def test_project_registry_covers_every_script_with_unique_stable_port() -> None:
assert video_environment["GUANGHE_LUGGAGE_USER_DATA_DIR"].endswith(
"-luggage"
)
assert video_environment["GUANGHE_NEW_BAG_USER_DATA_DIR"].endswith(
"-new-bag"
)
assert video_environment["GUANGHE_NEW_BAG_USER_DATA_DIR"] != (
video_environment["GUANGHE_LUGGAGE_USER_DATA_DIR"]
)
jd_video_upload = first.binding_for("product.jd_video_upload.run")
assert jd_video_upload is first.binding_for(
"product_commerce:upload_video_to_jd.py"