# Extra Sections (GitHub Trending / Hacker News / Insights) Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add three new content sections (GitHub Trending, Hacker News, cross-section Insights) to the morning push, while keeping evening push as RSS-only and RSS as the core load-bearing module.

**Architecture:** Each section is an autonomous module under `src/sections/<board>/` returning `(markdown, error)`. `push_job` orchestrates four modules (RSS / GitHub / HN run in parallel via `asyncio.gather`; Insights runs after, consuming the three section outputs), wraps each output with HTML-comment sentinels (`<!-- SECTION:xxx BEGIN/END -->`), and writes the assembled push file. Module failure (other than RSS) degrades to omitting that section. State is local files only — `news-data/trending-history.json` deduplicates GitHub repos across days.

**Tech Stack:** Python 3.12 / asyncio / aiohttp / BeautifulSoup4 (new) / croniter / markdownify / DeepSeek (OpenAI-compatible) LLM API / Algolia HN public API / GitHub REST API v3.

**Spec reference:** `docs/extra-sections-design.md`

---

## Phase 0: Prerequisites

### Task 0: Add BeautifulSoup4 dependency

**Files:**
- Modify: `pyproject.toml`

- [ ] **Step 1: Add bs4 to dependencies**

Open `pyproject.toml` and add `"beautifulsoup4>=4.12.0",` to the `dependencies` list (after `"aiohttp>=3.9.0",`).

- [ ] **Step 2: Sync deps**

Run: `uv sync`
Expected: package installed, no errors.

- [ ] **Step 3: Verify import works**

Run: `uv run python -c "from bs4 import BeautifulSoup; print('ok')"`
Expected: `ok`

- [ ] **Step 4: Commit**

```bash
git add pyproject.toml uv.lock
git commit -m "build: add beautifulsoup4 for HTML parsing"
```

---

## Phase 1: Storage Layer Foundation

The storage layer adds sentinel-aware section extraction, a `TrendingHistory` class for GH dedup, and a `profile` field on push frontmatter. These changes are independent of any new section module and must land first.

### Task 1: Add `extract_section` to storage.py

Parses `<!-- SECTION:xxx BEGIN/END -->` boundaries. Backward-compatible: old push files without sentinel return whole body when `section="rss"`, empty otherwise.

**Files:**
- Modify: `src/storage.py` (append after `_extract_push_titles`)
- Test: `tests/pytest/test_storage_sections.py` (create)

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_storage_sections.py`:

```python
"""测试新增的 sentinel 切片与 section-aware 读取"""

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent.parent / "src"))

from storage import extract_section


class TestExtractSection:
    def test_extract_section_with_sentinel(self):
        md = (
            "intro\n"
            "<!-- SECTION:rss BEGIN -->\n"
            "RSS body\n"
            "<!-- SECTION:rss END -->\n"
            "\n"
            "<!-- SECTION:github BEGIN -->\n"
            "GH body\n"
            "<!-- SECTION:github END -->\n"
        )
        assert extract_section(md, "rss").strip() == "RSS body"
        assert extract_section(md, "github").strip() == "GH body"
        assert extract_section(md, "hackernews") == ""

    def test_extract_section_legacy_file_rss(self):
        legacy = "# AI Daily\n### 1️⃣ foo\n### 2️⃣ bar\n"
        assert extract_section(legacy, "rss") == legacy

    def test_extract_section_legacy_file_non_rss(self):
        legacy = "# AI Daily\n### 1️⃣ foo\n"
        assert extract_section(legacy, "github") == ""
        assert extract_section(legacy, "hackernews") == ""
        assert extract_section(legacy, "insights") == ""

    def test_extract_section_missing_end_marker(self):
        broken = "<!-- SECTION:rss BEGIN -->\ncontent only\n"
        assert extract_section(broken, "rss") == ""
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_storage_sections.py -v`
Expected: ImportError or `AttributeError: module 'storage' has no attribute 'extract_section'`

- [ ] **Step 3: Implement `extract_section`**

Append to `src/storage.py`:

```python
_SECTION_RE_CACHE: Dict[str, re.Pattern] = {}


def _section_re(section: str) -> re.Pattern:
    """获取/缓存 sentinel 正则。section 名做转义,允许字母数字下划线"""
    if section not in _SECTION_RE_CACHE:
        s = re.escape(section)
        pattern = rf"<!--\s*SECTION:{s}\s*BEGIN\s*-->(.*?)<!--\s*SECTION:{s}\s*END\s*-->"
        _SECTION_RE_CACHE[section] = re.compile(pattern, flags=re.DOTALL)
    return _SECTION_RE_CACHE[section]


def extract_section(push_md: str, section: str) -> str:
    """从 push 文件内容中切出 <!-- SECTION:{section} BEGIN/END --> 之间的 markdown。

    向后兼容:
    - 新文件(带 sentinel): 返回 sentinel 边界内的原文(不去边界空行)
    - 老文件(无 sentinel) 且 section == 'rss': 返回整个 push_md
    - 老文件(无 sentinel) 且 section != 'rss': 返回空字符串
    - sentinel 残缺(只有 BEGIN 没有 END): 返回空字符串
    """
    match = _section_re(section).search(push_md)
    if match:
        return match.group(1)

    # 老文件兜底:rss 段视为整个 body
    has_any_sentinel = "<!-- SECTION:" in push_md
    if section == "rss" and not has_any_sentinel:
        return push_md
    return ""
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_storage_sections.py::TestExtractSection -v`
Expected: 4 tests pass.

- [ ] **Step 5: Commit**

```bash
git add src/storage.py tests/pytest/test_storage_sections.py
git commit -m "feat(storage): add sentinel-aware extract_section with legacy fallback"
```

---

### Task 2: Add `load_recent_section_titles` to storage.py

Reuses the existing `_extract_push_titles` for the H3 scan, but scopes it to a single section via `extract_section`.

**Files:**
- Modify: `src/storage.py`
- Test: `tests/pytest/test_storage_sections.py` (append)

- [ ] **Step 1: Write the failing test**

Append to `tests/pytest/test_storage_sections.py`:

```python
from datetime import date, datetime, timedelta
from storage import load_recent_section_titles, save_push_file


class TestLoadRecentSectionTitles:
    def test_returns_empty_when_dir_missing(self, tmp_path):
        assert load_recent_section_titles("rss", 3, str(tmp_path / "missing")) == ""

    def test_returns_empty_when_no_files(self, tmp_path):
        assert load_recent_section_titles("rss", 3, str(tmp_path)) == ""

    def test_extracts_only_target_section_titles(self, tmp_path):
        from src.config import get_timezone

        today = datetime.now(get_timezone()).date()
        push_file = tmp_path / f"push-{today.isoformat()}-08-00-00.md"
        push_file.write_text(
            f'---\npushDate: "{datetime.now(get_timezone()).isoformat()}"\n---\n\n'
            "<!-- SECTION:rss BEGIN -->\n"
            "### 1️⃣ RSS Title One\n"
            "### 2️⃣ RSS Title Two\n"
            "<!-- SECTION:rss END -->\n\n"
            "<!-- SECTION:github BEGIN -->\n"
            "### GH Repo Title\n"
            "<!-- SECTION:github END -->\n",
            encoding="utf-8",
        )
        rss_titles = load_recent_section_titles("rss", 3, str(tmp_path))
        assert "RSS Title One" in rss_titles
        assert "RSS Title Two" in rss_titles
        assert "GH Repo Title" not in rss_titles

        gh_titles = load_recent_section_titles("github", 3, str(tmp_path))
        assert "GH Repo Title" in gh_titles
        assert "RSS Title One" not in gh_titles
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_storage_sections.py::TestLoadRecentSectionTitles -v`
Expected: `AttributeError` on the import.

- [ ] **Step 3: Implement `load_recent_section_titles`**

Append to `src/storage.py`:

```python
def load_recent_section_titles(
    section: str, days: int, data_dir: str = "news-data"
) -> str:
    """加载最近 days 天 push 文件中 section 段的标题清单(供 LLM 查重防风格趋同)。

    返回紧凑纯文本,每行一条事件;遇到老文件(无 sentinel)按 extract_section 的兜底语义处理。
    """
    data_path = Path(data_dir)
    if not data_path.exists():
        return ""

    tz = get_timezone()
    today = datetime.now(tz).date()

    items: List[tuple] = []
    loaded_files: List[str] = []
    for i in range(days):
        d = today - timedelta(days=i)
        pattern = f"push-{d.isoformat()}-*.md"
        for push_file in sorted(data_path.glob(pattern)):
            if push_file.stat().st_size == 0:
                continue
            try:
                with open(push_file, "r", encoding="utf-8") as f:
                    content = f.read()
            except Exception:
                continue
            section_md = extract_section(content, section)
            if not section_md:
                continue
            items.extend(_extract_push_titles(section_md))
            loaded_files.append(push_file.name)

    if loaded_files:
        print(
            f"   📂 已加载 {len(loaded_files)} 个 push 文件 (section={section}): "
            f"{', '.join(loaded_files)}"
        )

    return "\n".join(f"- [{t}] {title}" if t else f"- {title}" for t, title in items)
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_storage_sections.py::TestLoadRecentSectionTitles -v`
Expected: 3 tests pass.

- [ ] **Step 5: Commit**

```bash
git add src/storage.py tests/pytest/test_storage_sections.py
git commit -m "feat(storage): add load_recent_section_titles for section-scoped history"
```

---

### Task 3: Add `TrendingHistory` to storage.py

Owns `news-data/trending-history.json` read / write / touch / cleanup. Single source of truth for GH dedup state.

**Files:**
- Modify: `src/storage.py`
- Test: `tests/pytest/test_storage_sections.py` (append)

- [ ] **Step 1: Write the failing test**

Append to `tests/pytest/test_storage_sections.py`:

```python
from storage import TrendingHistory, load_trending_history


class TestTrendingHistory:
    def test_load_missing_file(self, tmp_path):
        path = tmp_path / "trending.json"
        h = load_trending_history(str(path))
        assert h.repos == {}

    def test_touch_then_save_then_reload(self, tmp_path):
        path = tmp_path / "trending.json"
        h = load_trending_history(str(path))
        today = date(2026, 5, 17)
        h.touch("https://github.com/a/b", today)
        h.touch("https://github.com/c/d", today)
        h.save()

        h2 = load_trending_history(str(path))
        assert h2.repos == {
            "https://github.com/a/b": "2026-05-17",
            "https://github.com/c/d": "2026-05-17",
        }

    def test_contains_returns_membership(self, tmp_path):
        h = load_trending_history(str(tmp_path / "x.json"))
        h.touch("https://github.com/a/b", date(2026, 5, 17))
        assert "https://github.com/a/b" in h
        assert "https://github.com/x/y" not in h

    def test_cleanup_removes_expired_entries(self, tmp_path):
        path = tmp_path / "trending.json"
        path.write_text(
            '{"repos": {'
            '"https://github.com/old/repo": "2026-05-01", '
            '"https://github.com/new/repo": "2026-05-15"'
            '}, "updated_at": "2026-05-15T00:00:00+08:00"}',
            encoding="utf-8",
        )
        h = load_trending_history(str(path))
        h.cleanup(today=date(2026, 5, 17), keep_days=7)
        assert "https://github.com/old/repo" not in h
        assert "https://github.com/new/repo" in h

    def test_cleanup_keeps_today_inclusive(self, tmp_path):
        h = load_trending_history(str(tmp_path / "x.json"))
        h.touch("https://github.com/a/b", date(2026, 5, 10))
        # 2026-05-10 + 7 days = 2026-05-17 (last_seen 2026-05-10 仍在 keep 区间)
        h.cleanup(today=date(2026, 5, 17), keep_days=7)
        assert "https://github.com/a/b" in h
        # 再过 1 天就出区间
        h.cleanup(today=date(2026, 5, 18), keep_days=7)
        assert "https://github.com/a/b" not in h
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_storage_sections.py::TestTrendingHistory -v`
Expected: import error.

- [ ] **Step 3: Implement `TrendingHistory` + loader**

Append to `src/storage.py`:

```python
class TrendingHistory:
    """GitHub trending 已查阅 repo 索引。

    repos 字段:url → last_seen_date (ISO YYYY-MM-DD)。
    每次早报 cleanup 一次,touch 完所有今日 URL 后 save。
    """

    def __init__(self, path: str, repos: Dict[str, str]):
        self._path = path
        self.repos: Dict[str, str] = dict(repos)

    def __contains__(self, url: str) -> bool:
        return url in self.repos

    def touch(self, url: str, today: date) -> None:
        self.repos[url] = today.isoformat()

    def cleanup(self, today: date, keep_days: int) -> None:
        cutoff = today - timedelta(days=keep_days)
        self.repos = {
            url: d
            for url, d in self.repos.items()
            if _parse_iso_date_safe(d) is not None
            and _parse_iso_date_safe(d) >= cutoff
        }

    def save(self) -> None:
        path = Path(self._path)
        path.parent.mkdir(parents=True, exist_ok=True)
        payload = {
            "repos": self.repos,
            "updated_at": datetime.now(get_timezone()).isoformat(),
        }
        with open(path, "w", encoding="utf-8") as f:
            json.dump(payload, f, ensure_ascii=False, indent=2)


def _parse_iso_date_safe(s: str) -> Optional[date]:
    try:
        return date.fromisoformat(s)
    except (ValueError, TypeError):
        return None


def load_trending_history(path: str) -> TrendingHistory:
    """读取 trending-history.json;不存在返回空实例。"""
    p = Path(path)
    if not p.exists() or p.stat().st_size == 0:
        return TrendingHistory(path, {})
    try:
        with open(p, "r", encoding="utf-8") as f:
            data = json.load(f)
        return TrendingHistory(path, data.get("repos", {}))
    except (json.JSONDecodeError, OSError):
        print(f"⚠️ trending-history 读取失败,使用空索引: {path}")
        return TrendingHistory(path, {})
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_storage_sections.py::TestTrendingHistory -v`
Expected: 5 tests pass.

- [ ] **Step 5: Commit**

```bash
git add src/storage.py tests/pytest/test_storage_sections.py
git commit -m "feat(storage): add TrendingHistory for GH repo dedup state"
```

---

### Task 4: Add `profile` field to `save_push_file`

Push frontmatter gets a `profile: "morning"|"default"` tag so downstream tools can filter.

**Files:**
- Modify: `src/storage.py:363-378` (the existing `save_push_file`)
- Test: `tests/pytest/test_storage_sections.py` (append)

- [ ] **Step 1: Write the failing test**

Append to `tests/pytest/test_storage_sections.py`:

```python
from storage import save_push_file


class TestSavePushFileProfile:
    def test_default_profile_when_not_specified(self, tmp_path):
        f = tmp_path / "push-x.md"
        save_push_file(str(f), "body content", source_count=1, total_entries=1)
        text = f.read_text(encoding="utf-8")
        assert 'profile: "default"' in text
        assert "body content" in text

    def test_morning_profile(self, tmp_path):
        f = tmp_path / "push-x.md"
        save_push_file(
            str(f), "body", source_count=2, total_entries=3, profile="morning"
        )
        text = f.read_text(encoding="utf-8")
        assert 'profile: "morning"' in text
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_storage_sections.py::TestSavePushFileProfile -v`
Expected: FAIL (current `save_push_file` doesn't accept `profile`).

- [ ] **Step 3: Update `save_push_file` signature**

Open `src/storage.py`. Replace the `save_push_file` function (currently around line 363-378) with:

```python
def save_push_file(
    filepath: str,
    content: str,
    source_count: int,
    total_entries: int,
    profile: str = "default",
):
    """保存推送文件（Markdown格式）

    Args:
        profile: "morning" | "default"  ← 早报或常规;写入 frontmatter,便于按 profile 分析
    """
    path = Path(filepath)
    path.parent.mkdir(parents=True, exist_ok=True)

    push_time = datetime.now(get_timezone())
    frontmatter = (
        "---\n"
        f'pushDate: "{push_time.isoformat()}"\n'
        f'profile: "{profile}"\n'
        f"sourceCount: {source_count}\n"
        f"totalEntries: {total_entries}\n"
        "---\n\n"
    )

    with open(path, "w", encoding="utf-8") as f:
        f.write(frontmatter + content)
```

- [ ] **Step 4: Verify existing storage tests still pass**

Run: `uv run pytest tests/pytest/test_storage.py tests/pytest/test_storage_sections.py -v`
Expected: all pass (existing callers pass no `profile`, default "default" kicks in).

- [ ] **Step 5: Commit**

```bash
git add src/storage.py tests/pytest/test_storage_sections.py
git commit -m "feat(storage): add profile field to push frontmatter"
```

---

### Task 5: Extend `cleanup_old_files` to prune expired entries in `trending-history.json`

The fetch job's daily cleanup should not delete the trending-history file (it's cumulative state), but should remove entries older than `keep_days`.

**Files:**
- Modify: `src/storage.py:419-451` (the existing `cleanup_old_files`)
- Test: `tests/pytest/test_storage_sections.py` (append)

- [ ] **Step 1: Write the failing test**

Append to `tests/pytest/test_storage_sections.py`:

```python
from storage import cleanup_old_files


class TestCleanupOldFilesTrendingHistory:
    def test_prunes_trending_history_entries_not_file(self, tmp_path):
        path = tmp_path / "trending-history.json"
        old_date = (datetime.now().date() - timedelta(days=30)).isoformat()
        fresh_date = datetime.now().date().isoformat()
        path.write_text(
            '{"repos": {'
            f'"https://github.com/a/b": "{old_date}", '
            f'"https://github.com/c/d": "{fresh_date}"'
            '}, "updated_at": "..."}',
            encoding="utf-8",
        )
        cleanup_old_files(days=7, data_dir=str(tmp_path))
        # 文件应保留
        assert path.exists()
        # 过期条目应被剪枝
        import json as _j
        data = _j.loads(path.read_text(encoding="utf-8"))
        assert "https://github.com/a/b" not in data["repos"]
        assert "https://github.com/c/d" in data["repos"]
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_storage_sections.py::TestCleanupOldFilesTrendingHistory -v`
Expected: FAIL (current cleanup doesn't touch trending-history).

- [ ] **Step 3: Update `cleanup_old_files`**

Open `src/storage.py`. After the existing `for pattern in [...]` loop (around line 428-447), but **before** the final `if deleted_count > 0:` print, insert:

```python
    # trending-history.json: 剪枝过期条目,保留文件本身
    trending_path = data_path / "trending-history.json"
    if trending_path.exists() and trending_path.stat().st_size > 0:
        try:
            history = load_trending_history(str(trending_path))
            before = len(history.repos)
            history.cleanup(today=datetime.now().date(), keep_days=days)
            after = len(history.repos)
            if after < before:
                history.save()
                print(f"   ✂️ trending-history 剪枝: {before} → {after} 条")
        except Exception as e:
            print(f"   ⚠️ trending-history 剪枝失败: {e}")
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_storage_sections.py::TestCleanupOldFilesTrendingHistory tests/pytest/test_storage.py -v`
Expected: all pass.

- [ ] **Step 5: Commit**

```bash
git add src/storage.py tests/pytest/test_storage_sections.py
git commit -m "feat(storage): prune expired entries in trending-history.json on cleanup"
```

---

## Phase 2: RSS Module Migration

Migrate the existing RSS push flow into the new `sections` package without changing behavior.

### Task 6: Create `src/sections/rss/section.py`

`run_rss_section(config, now)` wraps existing `collect_entries_for_push` + `compose_digest` and returns `(markdown, error)`.

**Files:**
- Create: `src/sections/__init__.py` (empty)
- Create: `src/sections/rss/__init__.py` (empty)
- Create: `src/sections/rss/section.py`
- Test: `tests/pytest/test_sections_rss.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_sections_rss.py`:

```python
"""测试 RSS 模块返回 (markdown, error) 契约"""

import sys
from pathlib import Path
from unittest.mock import patch, AsyncMock

import pytest

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.sections.rss.section import run_rss_section


@pytest.mark.asyncio
async def test_returns_markdown_when_entries_present(sample_config, tmp_path):
    with patch(
        "src.sections.rss.section.collect_entries_for_push",
        return_value=([{"link": "x", "title": "t", "score": 80}], []),
    ), patch(
        "src.sections.rss.section.compose_digest",
        new=AsyncMock(return_value="# digest body"),
    ), patch(
        "src.sections.rss.section.load_recent_push_titles", return_value=""
    ), patch(
        "src.sections.rss.section.get_last_push_file", return_value=None
    ):
        md, err = await run_rss_section(sample_config, now=None)

    assert md == "# digest body"
    assert err is None


@pytest.mark.asyncio
async def test_returns_empty_when_no_entries(sample_config):
    with patch(
        "src.sections.rss.section.collect_entries_for_push", return_value=([], [])
    ), patch(
        "src.sections.rss.section.get_last_push_file", return_value=None
    ):
        md, err = await run_rss_section(sample_config, now=None)

    assert md == ""
    assert err is None


@pytest.mark.asyncio
async def test_returns_error_on_compose_failure(sample_config):
    with patch(
        "src.sections.rss.section.collect_entries_for_push",
        return_value=([{"link": "x"}], []),
    ), patch(
        "src.sections.rss.section.compose_digest",
        new=AsyncMock(side_effect=RuntimeError("LLM down")),
    ), patch(
        "src.sections.rss.section.load_recent_push_titles", return_value=""
    ), patch(
        "src.sections.rss.section.get_last_push_file", return_value=None
    ):
        md, err = await run_rss_section(sample_config, now=None)

    assert md == ""
    assert "LLM down" in err
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_sections_rss.py -v`
Expected: ModuleNotFoundError.

- [ ] **Step 3: Implement module**

Create `src/sections/__init__.py`:

```python
"""板块模块包。每个子模块导出 run_<board>_section(config, now) -> (markdown, error)"""
```

Create `src/sections/rss/__init__.py`:

```python
from src.sections.rss.section import run_rss_section

__all__ = ["run_rss_section"]
```

Create `src/sections/rss/section.py`:

```python
"""RSS 板块:沿用现有 collect_entries_for_push + compose_digest 流程

迁移自 src/main.py::run_push_job 中 RSS digest 部分,行为保持一致。
"""

from datetime import datetime
from typing import Dict, Optional, Tuple

from src.llm import compose_digest
from src.storage import (
    extract_push_time,
    get_last_push_file,
    load_recent_push_titles,
)


async def run_rss_section(
    config: Dict, now: Optional[datetime] = None
) -> Tuple[str, Optional[str]]:
    """生成 RSS digest markdown 段(不含 sentinel)。

    返回:
        (markdown, error)
        - 无新内容时返回 ("", None)
        - compose_digest 失败时返回 ("", error_message)
    """
    # 延迟 import 避免循环引用:collect_entries_for_push 仍在 main.py
    from src.main import collect_entries_for_push

    last_push_file = get_last_push_file()
    last_push_time = extract_push_time(last_push_file) if last_push_file else None

    min_score = config["filter"]["min_score"]
    context_days = config["filter"]["context_days"]

    to_push, context = collect_entries_for_push(
        last_push_time=last_push_time,
        context_days=context_days,
        min_score=min_score,
    )

    if not to_push:
        print("ℹ️ RSS: 无新消息")
        return "", None

    push_context_days = config["filter"].get("push_context_days", 5)
    recent = load_recent_push_titles(push_context_days)

    try:
        md = await compose_digest(to_push, context, config["llm"], recent_push_context=recent)
        return md, None
    except Exception as e:
        msg = f"compose_digest 失败: {e}"
        print(f"⚠️ RSS: {msg}")
        return "", msg
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_sections_rss.py -v`
Expected: 3 tests pass.

- [ ] **Step 5: Commit**

```bash
git add src/sections/ tests/pytest/test_sections_rss.py
git commit -m "feat(sections): extract RSS digest flow into run_rss_section"
```

---

## Phase 3: GitHub Module

### Task 7: Save a real GitHub trending HTML fixture

Save a real snapshot for deterministic parser tests. The structure may drift; this is the canonical "what we built against".

**Files:**
- Create: `tests/pytest/fixtures/github_trending.html`

- [ ] **Step 1: Download a real page**

```bash
mkdir -p tests/pytest/fixtures
curl -fsSL -A "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" \
  "https://github.com/trending" \
  -o tests/pytest/fixtures/github_trending.html
```

- [ ] **Step 2: Verify the fixture has expected markers**

Run: `grep -c 'class="Box-row"' tests/pytest/fixtures/github_trending.html`
Expected: a number ≥ 10 (typically 25). If 0, GitHub markup changed — adjust selectors in Task 8 accordingly.

- [ ] **Step 3: Commit fixture**

```bash
git add tests/pytest/fixtures/github_trending.html
git commit -m "test(github): snapshot github.com/trending fixture"
```

---

### Task 8: Implement `trending_scraper.py`

Async `fetch_trending_page` + `parse_trending_html` returning `[{url, full_name, description, language, stars_today, stars_total}]`.

**Files:**
- Create: `src/sections/github/__init__.py` (empty)
- Create: `src/sections/github/trending_scraper.py`
- Test: `tests/pytest/test_sections_github_scraper.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_sections_github_scraper.py`:

```python
"""测试 GitHub trending HTML 解析"""

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.sections.github.trending_scraper import parse_trending_html


def test_parse_trending_html_returns_repo_dicts():
    fixture = (
        Path(__file__).parent / "fixtures" / "github_trending.html"
    ).read_text(encoding="utf-8")

    repos = parse_trending_html(fixture)

    assert len(repos) > 0
    first = repos[0]
    assert first["url"].startswith("https://github.com/")
    assert "/" in first["full_name"]
    assert isinstance(first["stars_today"], int)
    assert isinstance(first["stars_total"], int)
    # description / language 可为空字符串但必须是 str
    assert isinstance(first["description"], str)
    assert isinstance(first["language"], str)


def test_parse_trending_html_dedupes_by_url():
    fixture = (
        Path(__file__).parent / "fixtures" / "github_trending.html"
    ).read_text(encoding="utf-8")
    repos = parse_trending_html(fixture)
    urls = [r["url"] for r in repos]
    assert len(urls) == len(set(urls))


def test_parse_trending_html_empty_input():
    assert parse_trending_html("") == []
    assert parse_trending_html("<html><body>no repos</body></html>") == []
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_sections_github_scraper.py -v`
Expected: ModuleNotFoundError.

- [ ] **Step 3: Implement scraper**

Create `src/sections/github/__init__.py` (empty):

```python
```

Create `src/sections/github/trending_scraper.py`:

```python
"""GitHub Trending 单页 HTML 抓取与解析。

数据源: https://github.com/trending (无 language / since 过滤)
"""

import re
from typing import Dict, List

import aiohttp
from bs4 import BeautifulSoup

TRENDING_URL = "https://github.com/trending"
USER_AGENT = (
    "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
    "(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
)
_NUM_RE = re.compile(r"[\d,]+")


def _parse_int(s: str) -> int:
    m = _NUM_RE.search(s or "")
    if not m:
        return 0
    return int(m.group(0).replace(",", ""))


async def fetch_trending_page(timeout: int = 10) -> str:
    """抓取 trending 页 HTML;非 200 抛 RuntimeError"""
    async with aiohttp.ClientSession(
        headers={"User-Agent": USER_AGENT}
    ) as session:
        async with session.get(
            TRENDING_URL, timeout=aiohttp.ClientTimeout(total=timeout)
        ) as resp:
            if resp.status != 200:
                raise RuntimeError(
                    f"GitHub trending 返回 {resp.status}: {await resp.text()[:200]}"
                )
            return await resp.text()


def parse_trending_html(html: str) -> List[Dict]:
    """解析 trending HTML,返回去重后的 repo 字典数组。"""
    if not html:
        return []

    soup = BeautifulSoup(html, "html.parser")
    seen_urls = set()
    repos: List[Dict] = []

    for article in soup.select("article.Box-row"):
        h2 = article.find("h2")
        a = h2.find("a") if h2 else None
        if not a or not a.get("href"):
            continue

        href = a["href"].strip()
        full_name = href.lstrip("/")
        url = f"https://github.com/{full_name}"
        if url in seen_urls:
            continue
        seen_urls.add(url)

        # description
        desc_tag = article.find("p")
        description = (desc_tag.get_text(strip=True) if desc_tag else "") or ""

        # language
        lang_tag = article.find("span", attrs={"itemprop": "programmingLanguage"})
        language = (lang_tag.get_text(strip=True) if lang_tag else "") or ""

        # stars_total: 第一个指向 /stargazers 的链接
        stars_total = 0
        star_a = article.find("a", href=re.compile(r"/stargazers$"))
        if star_a:
            stars_total = _parse_int(star_a.get_text(strip=True))

        # stars_today: 末尾的 "N stars today" span (class 多变,按文本)
        stars_today = 0
        for span in article.find_all("span"):
            t = span.get_text(strip=True)
            if "stars today" in t or "stars this week" in t or "stars this month" in t:
                stars_today = _parse_int(t)
                break

        repos.append(
            {
                "url": url,
                "full_name": full_name,
                "description": description,
                "language": language,
                "stars_today": stars_today,
                "stars_total": stars_total,
            }
        )

    return repos
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_sections_github_scraper.py -v`
Expected: 3 tests pass.

- [ ] **Step 5: Commit**

```bash
git add src/sections/github/ tests/pytest/test_sections_github_scraper.py
git commit -m "feat(github): add trending page scraper with HTML fixture test"
```

---

### Task 9: Implement `repo_enricher.py`

Two REST API calls per repo (`/repos/{o}/{r}` + `/repos/{o}/{r}/readme`). Token via env var optional.

**Files:**
- Create: `src/sections/github/repo_enricher.py`
- Test: `tests/pytest/test_sections_github_enricher.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_sections_github_enricher.py`:

```python
"""测试 GitHub REST API enrich 字段映射、archived 过滤、token 鉴权头"""

import base64
import os
import sys
from pathlib import Path
from unittest.mock import patch, AsyncMock, MagicMock

import pytest

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.sections.github.repo_enricher import enrich_repo, _auth_headers


def test_auth_headers_with_token(monkeypatch):
    monkeypatch.setenv("GITHUB_TOKEN", "ghp_secret")
    headers = _auth_headers(token_env="GITHUB_TOKEN")
    assert headers["Authorization"] == "Bearer ghp_secret"
    assert headers["Accept"] == "application/vnd.github+json"


def test_auth_headers_without_token(monkeypatch):
    monkeypatch.delenv("GITHUB_TOKEN", raising=False)
    headers = _auth_headers(token_env="GITHUB_TOKEN")
    assert "Authorization" not in headers
    assert headers["Accept"] == "application/vnd.github+json"


@pytest.mark.asyncio
async def test_enrich_repo_merges_metadata_and_readme():
    readme_body = "# Title\n\nProject description here."
    readme_b64 = base64.b64encode(readme_body.encode("utf-8")).decode("ascii")

    metadata_payload = {
        "description": "real desc",
        "topics": ["llm", "rag"],
        "license": {"spdx_id": "MIT"},
        "pushed_at": "2026-05-16T10:00:00Z",
        "archived": False,
    }
    readme_payload = {"content": readme_b64, "encoding": "base64"}

    async def fake_get_json(session, url, **kwargs):
        if url.endswith("/readme"):
            return readme_payload
        return metadata_payload

    base = {
        "url": "https://github.com/o/r",
        "full_name": "o/r",
        "description": "from trending",
        "language": "Python",
        "stars_today": 100,
        "stars_total": 5000,
    }

    with patch(
        "src.sections.github.repo_enricher._get_json", new=AsyncMock(side_effect=fake_get_json)
    ):
        enriched = await enrich_repo(
            session=MagicMock(), repo=base, token_env="GITHUB_TOKEN", readme_max_chars=200
        )

    assert enriched["topics"] == ["llm", "rag"]
    assert enriched["license"] == "MIT"
    assert enriched["pushed_at"] == "2026-05-16T10:00:00Z"
    assert "Project description" in enriched["readme_excerpt"]
    # trending 已有字段保留
    assert enriched["stars_today"] == 100


@pytest.mark.asyncio
async def test_enrich_repo_returns_none_when_archived():
    metadata_payload = {"archived": True, "topics": [], "pushed_at": "x"}

    async def fake_get_json(session, url, **kwargs):
        if url.endswith("/readme"):
            return {"content": ""}
        return metadata_payload

    base = {"url": "https://github.com/o/r", "full_name": "o/r"}
    with patch(
        "src.sections.github.repo_enricher._get_json", new=AsyncMock(side_effect=fake_get_json)
    ):
        result = await enrich_repo(
            session=MagicMock(), repo=base, token_env="GITHUB_TOKEN", readme_max_chars=200
        )
    assert result is None


@pytest.mark.asyncio
async def test_enrich_repo_truncates_readme():
    readme_body = "x" * 5000
    readme_b64 = base64.b64encode(readme_body.encode("utf-8")).decode("ascii")

    async def fake_get_json(session, url, **kwargs):
        if url.endswith("/readme"):
            return {"content": readme_b64, "encoding": "base64"}
        return {"archived": False, "topics": [], "pushed_at": "p"}

    base = {"url": "https://github.com/o/r", "full_name": "o/r"}
    with patch(
        "src.sections.github.repo_enricher._get_json", new=AsyncMock(side_effect=fake_get_json)
    ):
        enriched = await enrich_repo(
            session=MagicMock(), repo=base, token_env="GITHUB_TOKEN", readme_max_chars=100
        )
    assert len(enriched["readme_excerpt"]) == 100
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_sections_github_enricher.py -v`
Expected: ModuleNotFoundError.

- [ ] **Step 3: Implement enricher**

Create `src/sections/github/repo_enricher.py`:

```python
"""GitHub REST API enrich:metadata + README → enriched repo dict

匿名调用受 60 req/hr 限,设置 GITHUB_TOKEN 环境变量后走 5000 req/hr。
"""

import asyncio
import base64
import os
from typing import Dict, List, Optional, Tuple

import aiohttp

API_BASE = "https://api.github.com"


def _auth_headers(token_env: str = "GITHUB_TOKEN") -> Dict[str, str]:
    headers = {"Accept": "application/vnd.github+json"}
    token = os.environ.get(token_env)
    if token:
        headers["Authorization"] = f"Bearer {token}"
    return headers


async def _get_json(
    session: aiohttp.ClientSession, url: str, timeout: int = 10
) -> Optional[Dict]:
    async with session.get(
        url, timeout=aiohttp.ClientTimeout(total=timeout)
    ) as resp:
        if resp.status == 404:
            return None
        if resp.status != 200:
            raise RuntimeError(f"GitHub API {resp.status} for {url}")
        return await resp.json()


async def enrich_repo(
    session: aiohttp.ClientSession,
    repo: Dict,
    token_env: str = "GITHUB_TOKEN",
    readme_max_chars: int = 3000,
    timeout: int = 10,
) -> Optional[Dict]:
    """单 repo 双调用 enrich。返回 None 表示该 repo 应剔除(archived 或 metadata 不可达)。

    任一调用失败 raise → 调用方按 return_exceptions 模式聚合错误。
    """
    full_name = repo["full_name"]
    meta_url = f"{API_BASE}/repos/{full_name}"
    readme_url = f"{API_BASE}/repos/{full_name}/readme"

    meta, readme = await asyncio.gather(
        _get_json(session, meta_url, timeout=timeout),
        _get_json(session, readme_url, timeout=timeout),
    )

    if meta is None:
        return None
    if meta.get("archived"):
        return None

    license_spdx = ""
    if isinstance(meta.get("license"), dict):
        license_spdx = meta["license"].get("spdx_id") or ""

    readme_excerpt = ""
    if readme and readme.get("content"):
        try:
            raw = base64.b64decode(readme["content"]).decode("utf-8", errors="replace")
            readme_excerpt = raw[:readme_max_chars]
        except Exception:
            readme_excerpt = ""

    return {
        **repo,
        "topics": meta.get("topics") or [],
        "license": license_spdx,
        "pushed_at": meta.get("pushed_at") or "",
        "readme_excerpt": readme_excerpt,
    }


async def enrich_repos(
    candidates: List[Dict],
    token_env: str = "GITHUB_TOKEN",
    readme_max_chars: int = 3000,
    timeout: int = 10,
) -> Tuple[List[Dict], List[str]]:
    """并发 enrich 多个 repo。返回 (enriched_list_with_archived_filtered, errors)"""
    errors: List[str] = []
    headers = _auth_headers(token_env)

    async with aiohttp.ClientSession(headers=headers) as session:
        results = await asyncio.gather(
            *[
                enrich_repo(session, r, token_env, readme_max_chars, timeout)
                for r in candidates
            ],
            return_exceptions=True,
        )

    enriched: List[Dict] = []
    for r, candidate in zip(results, candidates):
        if isinstance(r, Exception):
            errors.append(f"enrich {candidate['full_name']} 失败: {r}")
        elif r is not None:
            enriched.append(r)
    return enriched, errors
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_sections_github_enricher.py -v`
Expected: 5 tests pass.

- [ ] **Step 5: Commit**

```bash
git add src/sections/github/repo_enricher.py tests/pytest/test_sections_github_enricher.py
git commit -m "feat(github): add repo_enricher using REST API metadata + readme"
```

---

### Task 10: Implement `src/sections/github/section.py`

Glue: scrape → cleanup history → filter → touch new → save history → enrich → LLM summarize.

**Files:**
- Create: `src/sections/github/section.py`
- Test: `tests/pytest/test_sections_github_section.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_sections_github_section.py`:

```python
"""测试 GitHub 板块编排:抓取 → history 过滤 → enrich → LLM 总结"""

import sys
from datetime import date
from pathlib import Path
from unittest.mock import patch, AsyncMock

import pytest

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.sections.github.section import run_github_section


def _cfg(history_file: str, max_deep_dive: int = 10) -> dict:
    return {
        "filter": {"keep_days": 7},
        "sections": {
            "github_trending": {
                "enabled": True,
                "max_items": 3,
                "max_deep_dive": max_deep_dive,
                "readme_max_chars": 3000,
                "history_file": history_file,
                "request_timeout": 10,
                "tokenName": "GITHUB_TOKEN",
            }
        },
        "llm": {
            "model": "x",
            "baseUrl": "http://x",
            "apiKeyName": "DEEPSEEK_API_KEY",
            "prompts": {"section_github": "prompts/section_github.md"},
        },
    }


@pytest.mark.asyncio
async def test_disabled_returns_empty(tmp_path):
    cfg = _cfg(str(tmp_path / "h.json"))
    cfg["sections"]["github_trending"]["enabled"] = False
    md, err = await run_github_section(cfg, now=None)
    assert md == ""
    assert err is None


@pytest.mark.asyncio
async def test_no_candidates_after_history_returns_empty(tmp_path):
    history_path = tmp_path / "h.json"
    # 预置 history,使得所有今日 scrape 出来的 repo 都已存在
    history_path.write_text(
        '{"repos": {"https://github.com/a/b": "2026-05-16"}, "updated_at": "x"}',
        encoding="utf-8",
    )
    cfg = _cfg(str(history_path))

    with patch(
        "src.sections.github.section.fetch_trending_page", new=AsyncMock(return_value="<html>")
    ), patch(
        "src.sections.github.section.parse_trending_html",
        return_value=[{"url": "https://github.com/a/b", "full_name": "a/b"}],
    ):
        md, err = await run_github_section(cfg, now=None)

    assert md == ""
    assert err is None


@pytest.mark.asyncio
async def test_happy_path_enriches_and_summarizes(tmp_path):
    history_path = tmp_path / "h.json"
    cfg = _cfg(str(history_path), max_deep_dive=10)

    repos = [
        {
            "url": "https://github.com/o1/r1",
            "full_name": "o1/r1",
            "description": "d1",
            "language": "Python",
            "stars_today": 100,
            "stars_total": 1000,
        }
    ]
    enriched = [{**repos[0], "topics": ["llm"], "license": "MIT", "pushed_at": "p", "readme_excerpt": "rm"}]

    with patch(
        "src.sections.github.section.fetch_trending_page", new=AsyncMock(return_value="<html>")
    ), patch(
        "src.sections.github.section.parse_trending_html", return_value=repos
    ), patch(
        "src.sections.github.section.enrich_repos",
        new=AsyncMock(return_value=(enriched, [])),
    ), patch(
        "src.sections.github.section.summarize_github_trending",
        new=AsyncMock(return_value=("## GH section md", None)),
    ):
        md, err = await run_github_section(cfg, now=None)

    assert md == "## GH section md"
    assert err is None
    # history 应已写入今日 scrape 出的 URL
    import json as _j
    saved = _j.loads(history_path.read_text(encoding="utf-8"))
    assert "https://github.com/o1/r1" in saved["repos"]


@pytest.mark.asyncio
async def test_truncates_candidates_to_max_deep_dive(tmp_path):
    cfg = _cfg(str(tmp_path / "h.json"), max_deep_dive=2)
    repos = [
        {"url": f"https://github.com/o/r{i}", "full_name": f"o/r{i}"} for i in range(5)
    ]
    captured = {}

    async def fake_enrich(candidates, **kwargs):
        captured["count"] = len(candidates)
        return [], []

    with patch(
        "src.sections.github.section.fetch_trending_page", new=AsyncMock(return_value="<html>")
    ), patch(
        "src.sections.github.section.parse_trending_html", return_value=repos
    ), patch(
        "src.sections.github.section.enrich_repos", new=AsyncMock(side_effect=fake_enrich)
    ):
        await run_github_section(cfg, now=None)

    assert captured["count"] == 2


@pytest.mark.asyncio
async def test_scrape_failure_returns_error(tmp_path):
    cfg = _cfg(str(tmp_path / "h.json"))
    with patch(
        "src.sections.github.section.fetch_trending_page",
        new=AsyncMock(side_effect=RuntimeError("HTTP 500")),
    ):
        md, err = await run_github_section(cfg, now=None)
    assert md == ""
    assert "HTTP 500" in err
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_sections_github_section.py -v`
Expected: ModuleNotFoundError.

- [ ] **Step 3: Implement section orchestrator**

Create `src/sections/github/section.py`:

```python
"""GitHub Trending 板块入口。

流程:trending 抓取 → history 过滤 → 候选写回 history → deep-dive → LLM 总结
"""

from datetime import datetime
from typing import Dict, Optional, Tuple

from src.config import get_timezone
from src.sections.github.repo_enricher import enrich_repos
from src.sections.github.trending_scraper import (
    fetch_trending_page,
    parse_trending_html,
)
from src.storage import load_trending_history


async def run_github_section(
    config: Dict, now: Optional[datetime] = None
) -> Tuple[str, Optional[str]]:
    cfg = config.get("sections", {}).get("github_trending", {})
    if not cfg.get("enabled", False):
        return "", None

    # 延迟 import 避免循环
    from src.llm import summarize_github_trending

    today = (now or datetime.now(get_timezone())).date()
    keep_days = config["filter"]["keep_days"]
    timeout = cfg.get("request_timeout", 10)
    max_deep_dive = cfg.get("max_deep_dive", 10)
    readme_max_chars = cfg.get("readme_max_chars", 3000)
    history_path = cfg.get("history_file", "news-data/trending-history.json")
    token_env = cfg.get("tokenName", "GITHUB_TOKEN")

    # 1. 抓取
    try:
        html = await fetch_trending_page(timeout=timeout)
    except Exception as e:
        return "", f"GH 抓取失败: {e}"

    all_repos = parse_trending_html(html)
    if not all_repos:
        return "", None

    # 2. history 加载 + 清理
    history = load_trending_history(history_path)
    history.cleanup(today=today, keep_days=keep_days)

    # 3. 候选筛选
    candidates = []
    for repo in all_repos:
        if repo["url"] in history:
            history.touch(repo["url"], today)
        else:
            candidates.append(repo)

    # 4. 候选写回 history + 持久化
    for repo in candidates:
        history.touch(repo["url"], today)
    history.save()

    if not candidates:
        return "", None
    if len(candidates) > max_deep_dive:
        candidates = candidates[:max_deep_dive]

    # 5. 并发 enrich
    enriched, enrich_errors = await enrich_repos(
        candidates,
        token_env=token_env,
        readme_max_chars=readme_max_chars,
        timeout=timeout,
    )
    for e in enrich_errors:
        print(f"⚠️ GH enrich: {e}")
    if not enriched:
        return "", None

    # 6. LLM 总结
    md, err = await summarize_github_trending(enriched, config["llm"])
    if err:
        return "", f"summarize_github_trending: {err}"
    return md or "", None
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_sections_github_section.py -v`
Expected: 5 tests pass.

- [ ] **Step 5: Commit**

```bash
git add src/sections/github/section.py tests/pytest/test_sections_github_section.py
git commit -m "feat(github): orchestrate trending → history → enrich → LLM"
```

---

### Task 11: Add `summarize_github_trending` to `src/llm.py` + `prompts/section_github.md`

**Files:**
- Modify: `src/llm.py` (append new async function)
- Create: `prompts/section_github.md`
- Test: `tests/pytest/test_sections_github_section.py` (already mocks the function; add direct test)

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_llm_extra_sections.py`:

```python
"""测试新增 LLM 函数 (summarize_github_trending 等)"""

import sys
from pathlib import Path
from unittest.mock import patch, AsyncMock

import pytest

sys.path.insert(0, str(Path(__file__).parent.parent.parent / "src"))

from llm import summarize_github_trending


@pytest.mark.asyncio
async def test_summarize_github_trending_happy_path(tmp_path):
    prompt_path = tmp_path / "section_github.md"
    prompt_path.write_text("Repos: {repos_json}\nmax_items={max_items}", encoding="utf-8")

    config = {
        "model": "x",
        "baseUrl": "http://x",
        "apiKeyName": "DEEPSEEK_API_KEY",
        "prompts": {"section_github": str(prompt_path)},
        "sections": {"github_trending": {"max_items": 3}},
    }
    enriched = [{"full_name": "o/r", "readme_excerpt": "rm"}]

    with patch("llm.call_llm", new=AsyncMock(return_value="## md")):
        md, err = await summarize_github_trending(enriched, config)

    assert md == "## md"
    assert err is None


@pytest.mark.asyncio
async def test_summarize_github_trending_llm_failure_returns_error(tmp_path):
    prompt_path = tmp_path / "section_github.md"
    prompt_path.write_text("x {repos_json} {max_items}", encoding="utf-8")
    config = {
        "model": "x",
        "baseUrl": "http://x",
        "apiKeyName": "DEEPSEEK_API_KEY",
        "prompts": {"section_github": str(prompt_path)},
        "sections": {"github_trending": {"max_items": 3}},
    }
    with patch("llm.call_llm", new=AsyncMock(side_effect=RuntimeError("boom"))):
        md, err = await summarize_github_trending([{"full_name": "o/r"}], config)
    assert md == ""
    assert "boom" in err
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_llm_extra_sections.py::test_summarize_github_trending_happy_path -v`
Expected: ImportError.

- [ ] **Step 3: Implement function + prompt**

Append to `src/llm.py`:

```python
async def summarize_github_trending(
    enriched_repos: List[Dict], config: Dict
) -> Tuple[str, Optional[str]]:
    """GH 板块总结:从 enriched 候选中选 1-max_items + 写 markdown。不传历史上下文。"""
    prompt_path = config.get("prompts", {}).get(
        "section_github", "prompts/section_github.md"
    )
    max_items = (
        config.get("sections", {}).get("github_trending", {}).get("max_items", 3)
    )
    prompt = load_prompt(
        prompt_path,
        repos_json=json.dumps(enriched_repos, ensure_ascii=False, indent=2),
        max_items=max_items,
    )
    try:
        return await call_llm(prompt, config), None
    except Exception as e:
        msg = f"summarize_github_trending 失败: {e}"
        print(f"⚠️ {msg}")
        return "", msg
```

Create `prompts/section_github.md`:

```markdown
你是开源情报分析师。从下列 GitHub Trending 候选项目中,挑出 **1-{max_items} 个**最值得关注的 AI 相关项目并行文。

## 关注领域(正面列表)
- **AI Agent**:智能体架构、工具链、多智能体、自主规划、Agent 框架
- **AI 模型**:训练、推理、微调、量化部署、模型服务、语音/多模态/视觉模型
- **AI 基础设施**:GPU 调度、芯片硬件、数据中心、推理优化、分布式训练、向量数据库、RAG 框架
- **大厂/前沿动态**:Apple、Google、Meta、OpenAI、Anthropic、Microsoft、xAI 等公司的官方动作与战略
- **AI 集成的开发者工具**:API 网关、自动化脚本、低代码平台等明确与 AI 协同的工具
- **创新性开源产品**:日增长显著且有清晰用户价值

## 排除(负面列表)
- 嵌入式开发(Arduino、ESP32、树莓派、单片机)
- 底层系统编程(内存分配器、编译器、链接器,与 AI 工作负载无关时)
- 通用开发工具(命名规范、代码风格、纯前端模板、UI 组件库、管理后台模板、静态网站主题)
- 学习资源(纯教程仓库、面试题合集、Roadmap,除非含实用代码的深度技术指南)
- 配置文件集合(Dotfiles、配置模板)
- 与 AI/科技无关的内容(电子书、资源搬运、刷榜项目)
- 纯娱乐/高风险误用(deepfake 等无明确基础设施价值)

## 输入数据
JSON 数组,字段:url / full_name / description / language / stars_today / stars_total / topics / license / pushed_at / readme_excerpt

```json
{repos_json}
```

## 选项规则
- 优先信号:`stars_today` 高 + `topics` 含 AI 标签(agent/llm/rag/inference/training 等) + readme 描述明确
- 跳过:`archived=true`(若漏过)、纯 awesome-list、个人 dotfiles
- 一句话价值定位需点明"解决什么问题",避免营销语("震撼""炸裂""革命性"等禁用)

## 输出格式(严格 Markdown,不要任何引导语)

```markdown
## ⭐ GitHub 趋势

- **owner/repo** ⭐{{stars_today}} — 一句话价值定位 [link]({{url}})
- ...
```

若候选中没有任何符合关注领域的项目,直接输出 `## ⭐ GitHub 趋势\n\n- 今日无显著 AI 相关趋势`,不要硬编。
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_llm_extra_sections.py::test_summarize_github_trending_happy_path tests/pytest/test_llm_extra_sections.py::test_summarize_github_trending_llm_failure_returns_error -v`
Expected: 2 pass.

- [ ] **Step 5: Commit**

```bash
git add src/llm.py prompts/section_github.md tests/pytest/test_llm_extra_sections.py
git commit -m "feat(llm): add summarize_github_trending with prompt"
```

---

## Phase 4: Hacker News Module

### Task 12: Save HN frontpage HTML fixture

**Files:**
- Create: `tests/pytest/fixtures/hn_frontpage.html`

- [ ] **Step 1: Download**

```bash
curl -fsSL -A "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 Chrome/120.0.0.0 Safari/537.36" \
  "https://news.ycombinator.com/news" \
  -o tests/pytest/fixtures/hn_frontpage.html
```

- [ ] **Step 2: Verify**

Run: `grep -c 'class="athing"' tests/pytest/fixtures/hn_frontpage.html`
Expected: ≥ 25 (usually 30).

- [ ] **Step 3: Commit**

```bash
git add tests/pytest/fixtures/hn_frontpage.html
git commit -m "test(hn): snapshot HN frontpage fixture"
```

---

### Task 13: Implement `frontpage_scraper.py`

**Files:**
- Create: `src/sections/hackernews/__init__.py` (empty)
- Create: `src/sections/hackernews/frontpage_scraper.py`
- Test: `tests/pytest/test_sections_hackernews_scraper.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_sections_hackernews_scraper.py`:

```python
"""测试 HN 首页 HTML 解析"""

import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.sections.hackernews.frontpage_scraper import parse_frontpage_html


def test_parse_frontpage_returns_stories():
    fixture = (
        Path(__file__).parent / "fixtures" / "hn_frontpage.html"
    ).read_text(encoding="utf-8")

    stories = parse_frontpage_html(fixture)

    assert len(stories) >= 25
    s = stories[0]
    assert s["id"]
    assert s["title"]
    assert s["url"]
    assert isinstance(s["points"], int)
    assert isinstance(s["comments"], int)
    assert s["comments_url"].startswith("https://news.ycombinator.com/item?id=")


def test_parse_frontpage_detects_show_hn_internal_url():
    # 构造一个最小内部链接故事 (Ask HN / Show HN)
    html = """
    <table>
      <tr class="athing" id="111">
        <td class="title">
          <span class="titleline">
            <a href="item?id=111">Ask HN: what's new?</a>
          </span>
        </td>
      </tr>
      <tr>
        <td class="subtext">
          <span class="subline">
            <span class="score">50 points</span>
            by <a href="user?id=alice">alice</a>
            <span class="age"><a href="item?id=111">2 hours ago</a></span>
            | <a href="item?id=111">5&nbsp;comments</a>
          </span>
        </td>
      </tr>
    </table>
    """
    stories = parse_frontpage_html(html)
    assert len(stories) == 1
    s = stories[0]
    assert s["id"] == "111"
    assert s["url"].startswith("https://news.ycombinator.com/item?id=")
    assert s["site"] == ""
    assert s["points"] == 50
    assert s["comments"] == 5
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_sections_hackernews_scraper.py -v`
Expected: ModuleNotFoundError.

- [ ] **Step 3: Implement**

Create `src/sections/hackernews/__init__.py` (empty).

Create `src/sections/hackernews/frontpage_scraper.py`:

```python
"""HN 首页 HTML 抓取与解析。

数据源: https://news.ycombinator.com/news (30 条)
"""

import re
from typing import Dict, List

import aiohttp
from bs4 import BeautifulSoup

FRONTPAGE_URL = "https://news.ycombinator.com/news"
USER_AGENT = (
    "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
    "(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
)
_NUM_RE = re.compile(r"\d+")


def _first_int(text: str) -> int:
    m = _NUM_RE.search(text or "")
    return int(m.group(0)) if m else 0


async def fetch_frontpage(timeout: int = 10) -> str:
    async with aiohttp.ClientSession(headers={"User-Agent": USER_AGENT}) as session:
        async with session.get(
            FRONTPAGE_URL, timeout=aiohttp.ClientTimeout(total=timeout)
        ) as resp:
            if resp.status != 200:
                raise RuntimeError(f"HN frontpage 返回 {resp.status}")
            return await resp.text()


def parse_frontpage_html(html: str) -> List[Dict]:
    """解析首页 HTML,返回 [{id, title, url, site, points, comments, comments_url}]"""
    if not html:
        return []
    soup = BeautifulSoup(html, "html.parser")
    stories: List[Dict] = []

    for athing in soup.select("tr.athing"):
        item_id = athing.get("id")
        if not item_id:
            continue

        title_a = athing.select_one("span.titleline > a")
        if not title_a:
            continue
        title = title_a.get_text(strip=True)
        href = title_a.get("href", "")
        # 内部链接(Ask HN / Show HN)
        if href.startswith("item?id="):
            url = f"https://news.ycombinator.com/{href}"
            site = ""
        else:
            url = href
            site_tag = athing.select_one("span.sitestr")
            site = site_tag.get_text(strip=True) if site_tag else ""

        # 同 id 的下一个 tr 是 subtext
        sub_tr = athing.find_next_sibling("tr")
        points = 0
        comments = 0
        comments_url = f"https://news.ycombinator.com/item?id={item_id}"
        if sub_tr:
            score = sub_tr.select_one("span.score")
            if score:
                points = _first_int(score.get_text(strip=True))
            # 最后一个 a[href^="item?id="] 是评论链接
            comment_a = None
            for a in sub_tr.find_all("a", href=re.compile(r"^item\?id=")):
                comment_a = a
            if comment_a:
                comments = _first_int(comment_a.get_text(strip=True))
                comments_url = f"https://news.ycombinator.com/{comment_a['href']}"

        stories.append(
            {
                "id": item_id,
                "title": title,
                "url": url,
                "site": site,
                "points": points,
                "comments": comments,
                "comments_url": comments_url,
            }
        )

    return stories
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_sections_hackernews_scraper.py -v`
Expected: 2 pass.

- [ ] **Step 5: Commit**

```bash
git add src/sections/hackernews/__init__.py src/sections/hackernews/frontpage_scraper.py tests/pytest/test_sections_hackernews_scraper.py
git commit -m "feat(hn): add frontpage scraper with fixture test"
```

---

### Task 14: Implement `item_enricher.py`

Algolia for comments + post text; `html_to_markdown` for external link content.

**Files:**
- Create: `src/sections/hackernews/item_enricher.py`
- Test: `tests/pytest/test_sections_hackernews_enricher.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_sections_hackernews_enricher.py`:

```python
"""测试 HN enrich(Algolia + 外链正文)"""

import sys
from pathlib import Path
from unittest.mock import patch, AsyncMock, MagicMock

import pytest

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.sections.hackernews.item_enricher import enrich_story


@pytest.mark.asyncio
async def test_enrich_external_link_story():
    story = {
        "id": "111",
        "title": "T",
        "url": "https://example.com/post",
        "site": "example.com",
        "points": 100,
        "comments": 5,
        "comments_url": "https://news.ycombinator.com/item?id=111",
    }
    algolia_payload = {
        "text": None,
        "children": [
            {"text": "<p>comment one</p>"},
            {"text": "<p>comment two</p>"},
            {"text": "<p>comment three</p>"},
            {"text": "<p>comment four</p>"},
        ],
    }

    async def fake_algolia(session, item_id, **kw):
        return algolia_payload

    async def fake_link(session, url, **kw):
        return "<html><body><p>link body</p></body></html>"

    with patch(
        "src.sections.hackernews.item_enricher._fetch_algolia_item",
        new=AsyncMock(side_effect=fake_algolia),
    ), patch(
        "src.sections.hackernews.item_enricher._fetch_url_html",
        new=AsyncMock(side_effect=fake_link),
    ):
        enriched = await enrich_story(
            session=MagicMock(),
            story=story,
            top_comments=3,
            comment_max_chars=500,
            link_content_max_chars=3000,
            algolia_base="https://hn.algolia.com/api/v1",
            timeout=10,
        )

    assert len(enriched["top_comments"]) == 3
    assert "comment one" in enriched["top_comments"][0]
    assert "link body" in enriched["link_content"]


@pytest.mark.asyncio
async def test_enrich_show_hn_uses_root_text_no_external_fetch():
    story = {
        "id": "222",
        "title": "Show HN: T",
        "url": "https://news.ycombinator.com/item?id=222",
        "site": "",
        "points": 200,
        "comments": 10,
        "comments_url": "https://news.ycombinator.com/item?id=222",
    }
    algolia_payload = {
        "text": "<p>post body text</p>",
        "children": [{"text": "<p>c1</p>"}],
    }
    link_calls = []

    async def fake_algolia(session, item_id, **kw):
        return algolia_payload

    async def fake_link(session, url, **kw):
        link_calls.append(url)
        return "should not be called"

    with patch(
        "src.sections.hackernews.item_enricher._fetch_algolia_item",
        new=AsyncMock(side_effect=fake_algolia),
    ), patch(
        "src.sections.hackernews.item_enricher._fetch_url_html",
        new=AsyncMock(side_effect=fake_link),
    ):
        enriched = await enrich_story(
            session=MagicMock(),
            story=story,
            top_comments=3,
            comment_max_chars=500,
            link_content_max_chars=3000,
            algolia_base="https://hn.algolia.com/api/v1",
            timeout=10,
        )

    assert link_calls == []
    assert "post body text" in enriched["link_content"]


@pytest.mark.asyncio
async def test_enrich_truncates_comments_and_link():
    story = {
        "id": "333",
        "title": "T",
        "url": "https://example.com/a",
        "site": "example.com",
        "points": 100,
        "comments": 2,
        "comments_url": "x",
    }
    long_comment = "<p>" + ("y" * 2000) + "</p>"
    long_link = "<html><body>" + ("z" * 5000) + "</body></html>"

    async def fake_algolia(session, item_id, **kw):
        return {"text": None, "children": [{"text": long_comment}]}

    async def fake_link(session, url, **kw):
        return long_link

    with patch(
        "src.sections.hackernews.item_enricher._fetch_algolia_item",
        new=AsyncMock(side_effect=fake_algolia),
    ), patch(
        "src.sections.hackernews.item_enricher._fetch_url_html",
        new=AsyncMock(side_effect=fake_link),
    ):
        enriched = await enrich_story(
            session=MagicMock(),
            story=story,
            top_comments=3,
            comment_max_chars=100,
            link_content_max_chars=200,
            algolia_base="https://hn.algolia.com/api/v1",
            timeout=10,
        )

    assert len(enriched["top_comments"][0]) <= 100
    assert len(enriched["link_content"]) <= 200


@pytest.mark.asyncio
async def test_enrich_failure_returns_partial():
    story = {
        "id": "444",
        "title": "T",
        "url": "https://example.com/x",
        "site": "example.com",
        "points": 100,
        "comments": 2,
        "comments_url": "x",
    }

    async def fake_algolia(session, item_id, **kw):
        raise RuntimeError("algolia down")

    async def fake_link(session, url, **kw):
        return "<html><body>ok</body></html>"

    with patch(
        "src.sections.hackernews.item_enricher._fetch_algolia_item",
        new=AsyncMock(side_effect=fake_algolia),
    ), patch(
        "src.sections.hackernews.item_enricher._fetch_url_html",
        new=AsyncMock(side_effect=fake_link),
    ):
        enriched = await enrich_story(
            session=MagicMock(),
            story=story,
            top_comments=3,
            comment_max_chars=500,
            link_content_max_chars=3000,
            algolia_base="https://hn.algolia.com/api/v1",
            timeout=10,
        )

    # 算法失败 → top_comments 留空,link_content 仍获取
    assert enriched["top_comments"] == []
    assert "ok" in enriched["link_content"]
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_sections_hackernews_enricher.py -v`
Expected: ModuleNotFoundError.

- [ ] **Step 3: Implement**

Create `src/sections/hackernews/item_enricher.py`:

```python
"""HN 单 story enrich:Algolia 评论 + 外链正文。

Algolia API: GET /api/v1/items/{id}
- root.text 是 Show HN / Ask HN 的 post 正文
- root.children[] 是顶层评论(按 HN ranking 排序)
"""

import asyncio
from typing import Dict, List, Optional, Tuple

import aiohttp

from src.processor import html_to_markdown

USER_AGENT = (
    "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
    "(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
)


def _is_internal_hn_url(url: str) -> bool:
    return url.startswith("https://news.ycombinator.com/item?id=")


async def _fetch_algolia_item(
    session: aiohttp.ClientSession, item_id: str, algolia_base: str, timeout: int
) -> Dict:
    url = f"{algolia_base}/items/{item_id}"
    async with session.get(
        url, timeout=aiohttp.ClientTimeout(total=timeout)
    ) as resp:
        if resp.status != 200:
            raise RuntimeError(f"Algolia /items/{item_id} 返回 {resp.status}")
        return await resp.json()


async def _fetch_url_html(
    session: aiohttp.ClientSession, url: str, timeout: int
) -> str:
    async with session.get(
        url, timeout=aiohttp.ClientTimeout(total=timeout)
    ) as resp:
        if resp.status != 200:
            raise RuntimeError(f"外链 {url} 返回 {resp.status}")
        return await resp.text()


async def enrich_story(
    session: aiohttp.ClientSession,
    story: Dict,
    top_comments: int,
    comment_max_chars: int,
    link_content_max_chars: int,
    algolia_base: str,
    timeout: int,
) -> Dict:
    """对单 story enrich。任一子任务失败 → 对应字段留空,不抛。"""
    item_id = story["id"]
    is_internal = _is_internal_hn_url(story["url"])

    # 并发:Algolia + 外链(仅外链类)
    tasks = [_fetch_algolia_item(session, item_id, algolia_base, timeout)]
    if not is_internal:
        tasks.append(_fetch_url_html(session, story["url"], timeout))

    results = await asyncio.gather(*tasks, return_exceptions=True)
    algolia_result = results[0]
    external_html_result = results[1] if not is_internal else None

    # 评论解析
    comments_list: List[str] = []
    post_text = ""
    if not isinstance(algolia_result, Exception) and algolia_result:
        post_text = algolia_result.get("text") or ""
        children = algolia_result.get("children") or []
        for child in children[:top_comments]:
            raw = (child or {}).get("text") or ""
            if not raw:
                continue
            md = html_to_markdown(raw)
            comments_list.append(md[:comment_max_chars])

    # link_content
    link_content = ""
    if is_internal:
        # Show HN / Ask HN:post 自身正文
        if post_text:
            link_content = html_to_markdown(post_text)[:link_content_max_chars]
    else:
        if not isinstance(external_html_result, Exception) and external_html_result:
            link_content = html_to_markdown(
                external_html_result, base_url=story["url"]
            )[:link_content_max_chars]

    return {
        **story,
        "link_content": link_content,
        "top_comments": comments_list,
    }


async def enrich_stories(
    stories: List[Dict],
    top_comments: int,
    comment_max_chars: int,
    link_content_max_chars: int,
    algolia_base: str = "https://hn.algolia.com/api/v1",
    timeout: int = 10,
) -> Tuple[List[Dict], List[str]]:
    """并发 enrich 多个 stories。"""
    errors: List[str] = []
    async with aiohttp.ClientSession(headers={"User-Agent": USER_AGENT}) as session:
        results = await asyncio.gather(
            *[
                enrich_story(
                    session,
                    s,
                    top_comments,
                    comment_max_chars,
                    link_content_max_chars,
                    algolia_base,
                    timeout,
                )
                for s in stories
            ],
            return_exceptions=True,
        )
    enriched: List[Dict] = []
    for r, src in zip(results, stories):
        if isinstance(r, Exception):
            errors.append(f"enrich story {src['id']} 失败: {r}")
        else:
            enriched.append(r)
    return enriched, errors
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_sections_hackernews_enricher.py -v`
Expected: 4 pass.

- [ ] **Step 5: Commit**

```bash
git add src/sections/hackernews/item_enricher.py tests/pytest/test_sections_hackernews_enricher.py
git commit -m "feat(hn): add item enricher using Algolia + html_to_markdown"
```

---

### Task 15: Implement `src/sections/hackernews/section.py`

Glue: scrape → light LLM select → enrich → LLM summarize.

**Files:**
- Create: `src/sections/hackernews/section.py`
- Test: `tests/pytest/test_sections_hackernews_section.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_sections_hackernews_section.py`:

```python
import sys
from pathlib import Path
from unittest.mock import patch, AsyncMock

import pytest

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.sections.hackernews.section import run_hackernews_section


def _cfg() -> dict:
    return {
        "filter": {"keep_days": 7},
        "sections": {
            "hackernews": {
                "enabled": True,
                "select_k": 1,
                "top_comments": 20,
                "comment_max_chars": 500,
                "link_content_max_chars": 3000,
                "request_timeout": 10,
                "algolia_base": "https://hn.algolia.com/api/v1",
            }
        },
        "llm": {
            "model": "x",
            "baseUrl": "http://x",
            "apiKeyName": "DEEPSEEK_API_KEY",
            "prompts": {
                "section_hackernews_select": "prompts/section_hackernews_select.md",
                "section_hackernews": "prompts/section_hackernews.md",
            },
        },
    }


@pytest.mark.asyncio
async def test_disabled_returns_empty():
    cfg = _cfg()
    cfg["sections"]["hackernews"]["enabled"] = False
    md, err = await run_hackernews_section(cfg, now=None)
    assert md == ""
    assert err is None


@pytest.mark.asyncio
async def test_select_empty_returns_silent():
    cfg = _cfg()
    with patch(
        "src.sections.hackernews.section.fetch_frontpage", new=AsyncMock(return_value="<html>")
    ), patch(
        "src.sections.hackernews.section.parse_frontpage_html",
        return_value=[{"id": "1", "title": "x"}],
    ), patch(
        "src.sections.hackernews.section.select_ai_related_hn",
        new=AsyncMock(return_value=([], None)),
    ):
        md, err = await run_hackernews_section(cfg, now=None)
    assert md == ""
    assert err is None


@pytest.mark.asyncio
async def test_happy_path():
    cfg = _cfg()
    front = [{"id": "1", "title": "AI thing", "url": "https://e.com/a", "site": "e.com", "points": 100, "comments": 5, "comments_url": "x"}]
    enriched = [{**front[0], "link_content": "body", "top_comments": ["c1"]}]

    with patch(
        "src.sections.hackernews.section.fetch_frontpage", new=AsyncMock(return_value="<html>")
    ), patch(
        "src.sections.hackernews.section.parse_frontpage_html", return_value=front
    ), patch(
        "src.sections.hackernews.section.select_ai_related_hn",
        new=AsyncMock(return_value=(["1"], None)),
    ), patch(
        "src.sections.hackernews.section.enrich_stories",
        new=AsyncMock(return_value=(enriched, [])),
    ), patch(
        "src.sections.hackernews.section.summarize_hackernews",
        new=AsyncMock(return_value=("## HN md", None)),
    ):
        md, err = await run_hackernews_section(cfg, now=None)
    assert md == "## HN md"
    assert err is None


@pytest.mark.asyncio
async def test_scrape_failure_returns_error():
    cfg = _cfg()
    with patch(
        "src.sections.hackernews.section.fetch_frontpage",
        new=AsyncMock(side_effect=RuntimeError("net")),
    ):
        md, err = await run_hackernews_section(cfg, now=None)
    assert md == ""
    assert "net" in err
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_sections_hackernews_section.py -v`
Expected: ModuleNotFoundError.

- [ ] **Step 3: Implement section**

Create `src/sections/hackernews/section.py`:

```python
"""HN 板块入口。流程:首页 → 轻 LLM 选 K → enrich → 最终 LLM 行文"""

from datetime import datetime
from typing import Dict, Optional, Tuple

from src.sections.hackernews.frontpage_scraper import (
    fetch_frontpage,
    parse_frontpage_html,
)
from src.sections.hackernews.item_enricher import enrich_stories


async def run_hackernews_section(
    config: Dict, now: Optional[datetime] = None
) -> Tuple[str, Optional[str]]:
    cfg = config.get("sections", {}).get("hackernews", {})
    if not cfg.get("enabled", False):
        return "", None

    from src.llm import select_ai_related_hn, summarize_hackernews

    timeout = cfg.get("request_timeout", 10)
    select_k = cfg.get("select_k", 1)
    top_comments = cfg.get("top_comments", 20)
    comment_max_chars = cfg.get("comment_max_chars", 500)
    link_content_max_chars = cfg.get("link_content_max_chars", 3000)
    algolia_base = cfg.get("algolia_base", "https://hn.algolia.com/api/v1")

    # 1. 抓首页
    try:
        html = await fetch_frontpage(timeout=timeout)
    except Exception as e:
        return "", f"HN 首页抓取失败: {e}"

    front = parse_frontpage_html(html)
    if not front:
        return "", None

    # 2. 轻 LLM 初筛
    selected_ids, select_err = await select_ai_related_hn(front, k=select_k, config=config["llm"])
    if select_err:
        return "", f"select_ai_related_hn: {select_err}"
    if not selected_ids:
        return "", None

    selected = [s for s in front if s["id"] in set(selected_ids)]
    if not selected:
        return "", None

    # 3. enrich
    enriched, enrich_errors = await enrich_stories(
        selected,
        top_comments=top_comments,
        comment_max_chars=comment_max_chars,
        link_content_max_chars=link_content_max_chars,
        algolia_base=algolia_base,
        timeout=timeout,
    )
    for e in enrich_errors:
        print(f"⚠️ HN enrich: {e}")
    if not enriched:
        return "", None

    # 4. LLM 总结
    md, err = await summarize_hackernews(enriched, config["llm"])
    if err:
        return "", f"summarize_hackernews: {err}"
    return md or "", None
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_sections_hackernews_section.py -v`
Expected: 4 pass.

- [ ] **Step 5: Commit**

```bash
git add src/sections/hackernews/section.py tests/pytest/test_sections_hackernews_section.py
git commit -m "feat(hn): orchestrate frontpage → select → enrich → LLM"
```

---

### Task 16: Add `select_ai_related_hn` + `summarize_hackernews` to `src/llm.py` + 2 prompt files

**Files:**
- Modify: `src/llm.py`
- Create: `prompts/section_hackernews_select.md`
- Create: `prompts/section_hackernews.md`
- Test: `tests/pytest/test_llm_extra_sections.py` (append)

- [ ] **Step 1: Write the failing test**

Append to `tests/pytest/test_llm_extra_sections.py`:

```python
from llm import select_ai_related_hn, summarize_hackernews


@pytest.mark.asyncio
async def test_select_ai_related_hn_parses_id_array(tmp_path):
    prompt_path = tmp_path / "select.md"
    prompt_path.write_text("k={k} candidates={candidates_json}", encoding="utf-8")
    config = {
        "model": "x",
        "baseUrl": "http://x",
        "apiKeyName": "DEEPSEEK_API_KEY",
        "prompts": {"section_hackernews_select": str(prompt_path)},
    }
    with patch("llm.call_llm", new=AsyncMock(return_value='["111", "222"]')):
        ids, err = await select_ai_related_hn(
            [{"id": "111"}, {"id": "222"}, {"id": "333"}], k=2, config=config
        )
    assert ids == ["111", "222"]
    assert err is None


@pytest.mark.asyncio
async def test_select_ai_related_hn_empty_array(tmp_path):
    prompt_path = tmp_path / "select.md"
    prompt_path.write_text("{k}{candidates_json}", encoding="utf-8")
    config = {
        "model": "x",
        "baseUrl": "http://x",
        "apiKeyName": "DEEPSEEK_API_KEY",
        "prompts": {"section_hackernews_select": str(prompt_path)},
    }
    with patch("llm.call_llm", new=AsyncMock(return_value="[]")):
        ids, err = await select_ai_related_hn([{"id": "1"}], k=1, config=config)
    assert ids == []
    assert err is None


@pytest.mark.asyncio
async def test_summarize_hackernews_happy(tmp_path):
    prompt_path = tmp_path / "hn.md"
    prompt_path.write_text("{stories_json}", encoding="utf-8")
    config = {
        "model": "x",
        "baseUrl": "http://x",
        "apiKeyName": "DEEPSEEK_API_KEY",
        "prompts": {"section_hackernews": str(prompt_path)},
    }
    with patch("llm.call_llm", new=AsyncMock(return_value="## HN summary")):
        md, err = await summarize_hackernews(
            [{"id": "1", "title": "t", "link_content": "x", "top_comments": []}], config
        )
    assert md == "## HN summary"
    assert err is None
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_llm_extra_sections.py -v -k "select_ai_related_hn or summarize_hackernews"`
Expected: ImportError.

- [ ] **Step 3: Implement functions + prompts**

Append to `src/llm.py`:

```python
async def select_ai_related_hn(
    candidates: List[Dict], k: int, config: Dict
) -> Tuple[List[str], Optional[str]]:
    """轻 LLM:从 HN 首页候选元数据中挑 k 个 AI 相关 id。

    输入候选只含 id/title/site/points/comments 字段(不含正文)。
    """
    prompt_path = config.get("prompts", {}).get(
        "section_hackernews_select", "prompts/section_hackernews_select.md"
    )
    slim = [
        {
            "id": c.get("id"),
            "title": c.get("title", ""),
            "site": c.get("site", ""),
            "points": c.get("points", 0),
            "comments": c.get("comments", 0),
        }
        for c in candidates
    ]
    prompt = load_prompt(
        prompt_path,
        k=k,
        candidates_json=json.dumps(slim, ensure_ascii=False, indent=2),
    )
    try:
        response = await call_llm(prompt, config)
    except Exception as e:
        msg = f"select_ai_related_hn 失败: {e}"
        print(f"⚠️ {msg}")
        return [], msg

    try:
        ids = _parse_llm_json_response(response)
    except ValueError as e:
        msg = f"select_ai_related_hn 解析失败: {e}"
        print(f"⚠️ {msg}")
        return [], msg

    if not isinstance(ids, list):
        return [], "select_ai_related_hn 返回非数组"
    return [str(x) for x in ids][:k], None


async def summarize_hackernews(
    enriched_stories: List[Dict], config: Dict
) -> Tuple[str, Optional[str]]:
    """对输入的 K 个 enriched stories 行文。不传历史上下文。"""
    prompt_path = config.get("prompts", {}).get(
        "section_hackernews", "prompts/section_hackernews.md"
    )
    prompt = load_prompt(
        prompt_path,
        stories_json=json.dumps(enriched_stories, ensure_ascii=False, indent=2),
    )
    try:
        return await call_llm(prompt, config), None
    except Exception as e:
        msg = f"summarize_hackernews 失败: {e}"
        print(f"⚠️ {msg}")
        return "", msg
```

Create `prompts/section_hackernews_select.md`:

```markdown
你是 HN 早间选题人。从下列 30 条 HN 首页元数据中,挑出 **{k}** 个最符合关注领域的 story id。

## 关注领域(正面列表)
- AI Agent(智能体架构、工具链、多智能体、自主规划)
- AI 模型(训练、推理、微调、应用、语音/多模态)
- AI 基础设施(芯片、硬件、数据中心、推理优化、向量数据库、RAG 框架)
- 大厂/前沿动态(Apple、Google、Meta、OpenAI、Anthropic、Microsoft、xAI)
- AI 集成的开发者工具(API 网关、自动化、低代码与 AI 协同)

## 排除(负面列表)
- 嵌入式开发(Arduino、ESP32、树莓派、单片机)
- 底层系统编程(内存分配器、编译器、链接器,与 AI 无关时)
- 通用开发工具(命名规范、代码风格)
- 与 AI/科技无关的内容

## 决策原则
- title + site 不足以确定 AI 相关时,**宁可漏选不可错选**(错选会让最终板块写出与 AI Daily 调性无关的内容)
- 若全部 30 条都不符合,返回空数组 `[]`

## 输入
```json
{candidates_json}
```

## 输出
**只输出 JSON 数组**,如:`["12345", "67890"]` 或 `[]`
严禁输出任何解释性文字、markdown 包装、或自然语言句子。
```

Create `prompts/section_hackernews.md`:

```markdown
你是 HN 早间编辑。对输入的 enriched stories **全部行文**(不再二次挑选)。

## 输入(JSON 数组,字段含 link_content 与 top_comments)
```json
{stories_json}
```

## 内容要求(每条 story)
1. 提炼原文 `link_content` 的核心(背景 / 要点 / 结论)
2. 汇总 `top_comments` 中有价值的观点(支持 / 反对 / 补充),不是简单复述
3. 若评论中出现明显反驳原文的观点,必须保留并标注

## 输出格式(严格 Markdown,不要任何引导语)

```markdown
## 🟧 Hacker News 热议

### {{title}} ({{points}} pts · {{comments}} comments)

**📌 内容总结**

- 要点 1
- 要点 2
- 要点 3(可选)

**💬 HN 讨论**

- 观点 1(含反对/补充)
- 观点 2(可选)

🔗 [原文]({{url}}) | [HN 讨论页]({{comments_url}})
```

## 风格约束
- 客观、犀利、克制
- 避免与 RSS digest 句式雷同;不做行业宏大叙事
- 禁用词汇:震撼、炸裂、革命性、现象级
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_llm_extra_sections.py -v`
Expected: 5 tests pass (including the earlier GH tests).

- [ ] **Step 5: Commit**

```bash
git add src/llm.py prompts/section_hackernews_select.md prompts/section_hackernews.md tests/pytest/test_llm_extra_sections.py
git commit -m "feat(llm): add select_ai_related_hn and summarize_hackernews with prompts"
```

---

## Phase 5: Insights Module

### Task 17: Implement `insights/section.py` + `generate_trend_insights` + prompt

**Files:**
- Create: `src/sections/insights/__init__.py` (empty)
- Create: `src/sections/insights/section.py`
- Modify: `src/llm.py`
- Create: `prompts/insights.md`
- Test: `tests/pytest/test_sections_insights.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_sections_insights.py`:

```python
import sys
from pathlib import Path
from unittest.mock import patch, AsyncMock

import pytest

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.sections.insights.section import run_insights_section


def _cfg() -> dict:
    return {
        "filter": {"push_context_days": 5},
        "sections": {"insights": {"enabled": True}},
        "llm": {
            "model": "x",
            "baseUrl": "http://x",
            "apiKeyName": "DEEPSEEK_API_KEY",
            "prompts": {"insights": "prompts/insights.md"},
        },
    }


@pytest.mark.asyncio
async def test_disabled_returns_empty():
    cfg = _cfg()
    cfg["sections"]["insights"]["enabled"] = False
    md, err = await run_insights_section("rss", "gh", "hn", cfg, now=None)
    assert md == ""
    assert err is None


@pytest.mark.asyncio
async def test_marks_empty_sections_for_llm():
    cfg = _cfg()
    captured = {}

    async def fake_gen(sections, recent_insights, config):
        captured["sections"] = sections
        return "insights md", None

    with patch(
        "src.sections.insights.section.load_recent_section_titles", return_value=""
    ), patch(
        "src.sections.insights.section.generate_trend_insights",
        new=AsyncMock(side_effect=fake_gen),
    ):
        md, err = await run_insights_section("", "gh md", "", cfg, now=None)

    assert md == "insights md"
    assert captured["sections"]["rss"] == "(本次无内容)"
    assert captured["sections"]["github"] == "gh md"
    assert captured["sections"]["hackernews"] == "(本次无内容)"
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_sections_insights.py -v`
Expected: ModuleNotFoundError.

- [ ] **Step 3: Implement**

Create `src/sections/insights/__init__.py` (empty).

Create `src/sections/insights/section.py`:

```python
"""Insights 板块:基于 RSS/GH/HN 三段成品 + 近 N 天 insights 历史做跨板块小结"""

from datetime import datetime
from typing import Dict, Optional, Tuple

from src.storage import load_recent_section_titles

EMPTY_MARKER = "(本次无内容)"


async def run_insights_section(
    rss_md: str,
    gh_md: str,
    hn_md: str,
    config: Dict,
    now: Optional[datetime] = None,
) -> Tuple[str, Optional[str]]:
    cfg = config.get("sections", {}).get("insights", {})
    if not cfg.get("enabled", False):
        return "", None

    from src.llm import generate_trend_insights

    days = config["filter"].get("push_context_days", 5)
    recent = load_recent_section_titles("insights", days)

    sections = {
        "rss": rss_md or EMPTY_MARKER,
        "github": gh_md or EMPTY_MARKER,
        "hackernews": hn_md or EMPTY_MARKER,
    }

    md, err = await generate_trend_insights(sections, recent, config["llm"])
    if err:
        return "", f"generate_trend_insights: {err}"
    return md or "", None
```

Append to `src/llm.py`:

```python
async def generate_trend_insights(
    sections: Dict[str, str], recent_insights: str, config: Dict
) -> Tuple[str, Optional[str]]:
    """输入三段成品 + 近期 insights 标题清单,返回洞察段 markdown。"""
    prompt_path = config.get("prompts", {}).get("insights", "prompts/insights.md")
    prompt = load_prompt(
        prompt_path,
        rss=sections.get("rss", ""),
        github=sections.get("github", ""),
        hackernews=sections.get("hackernews", ""),
        recent_insights=recent_insights or "",
    )
    try:
        return await call_llm(prompt, config), None
    except Exception as e:
        msg = f"generate_trend_insights 失败: {e}"
        print(f"⚠️ {msg}")
        return "", msg
```

Create `prompts/insights.md`:

```markdown
你是 AI 行业观察员。基于今日三段已生成的成品做一段**跨板块趋势小结**。

## 今日素材(三段成品)

### RSS 板块
{rss}

### GitHub 板块
{github}

### Hacker News 板块
{hackernews}

## 近 N 天 insights 段标题(仅供风格参考与防趋同,严禁措辞模仿)
<RECENT BEGIN>
{recent_insights}
<RECENT END>

## 任务
基于今日三段成品,做跨板块小结。可参考的切入角度(不必全部覆盖,按今日素材最突出的张力来组织):
- 跨板块的"交叉信号"(同一话题在 RSS / GH / HN 中同时出现)
- 与近几天对比"新升温"或"退潮"的关键词
- "反直觉发现"(违反常识、值得停下来想一下的一条)
- 行业结构信号(资本 / 监管 / 算力 / 应用层等)

## 输出要求
- 直接输出 markdown 段,不带任何引导语
- 起始用 `## 💡 今日洞察`
- 总长度 200-400 字之间
- 不要简单复述其他板块的具体新闻;洞察的价值在"连接"而非"清单"
- 避免与 RSS digest 句式雷同;禁用宏大叙事词汇("深水区""临界点""下半场""博弈"等)
- 每条洞察须能溯源到今日素材中的具体信号,而不是凭空总结
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_sections_insights.py -v`
Expected: 2 pass.

- [ ] **Step 5: Commit**

```bash
git add src/sections/insights/ prompts/insights.md src/llm.py tests/pytest/test_sections_insights.py
git commit -m "feat(insights): add cross-section trend summary module"
```

---

## Phase 6: push_job Orchestration

### Task 18: Add `is_morning_push` to `src/main.py`

**Files:**
- Modify: `src/main.py`
- Test: `tests/pytest/test_morning_detection.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_morning_detection.py`:

```python
"""测试早报判定:cron + 容差"""

import sys
from datetime import datetime, timedelta, timezone
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.main import is_morning_push


TZ = timezone(timedelta(hours=8))


def _cfg(cron, tol=5):
    return {
        "schedule": {
            "morning_cron": cron,
            "morning_match_tolerance_minutes": tol,
            "timezone_hours": 8,
        }
    }


def test_returns_false_when_no_morning_cron_configured():
    assert is_morning_push(datetime(2026, 5, 17, 8, 0, tzinfo=TZ), {"schedule": {}}) is False


def test_match_exact_time():
    cfg = _cfg("0 8 * * *")
    now = datetime(2026, 5, 17, 8, 0, tzinfo=TZ)
    assert is_morning_push(now, cfg) is True


def test_match_within_tolerance():
    cfg = _cfg("0 8 * * *", tol=5)
    assert is_morning_push(datetime(2026, 5, 17, 8, 4, tzinfo=TZ), cfg) is True
    assert is_morning_push(datetime(2026, 5, 17, 7, 56, tzinfo=TZ), cfg) is True


def test_outside_tolerance():
    cfg = _cfg("0 8 * * *", tol=5)
    assert is_morning_push(datetime(2026, 5, 17, 8, 6, tzinfo=TZ), cfg) is False
    assert is_morning_push(datetime(2026, 5, 17, 9, 0, tzinfo=TZ), cfg) is False


def test_evening_time_not_morning():
    cfg = _cfg("0 8 * * *", tol=5)
    assert is_morning_push(datetime(2026, 5, 17, 17, 0, tzinfo=TZ), cfg) is False
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_morning_detection.py -v`
Expected: ImportError.

- [ ] **Step 3: Implement**

Open `src/main.py`. After the existing `calculate_push_times` function (around line 96), insert:

```python
def is_morning_push(now: datetime, config: Dict) -> bool:
    """判定当前时刻是否为早报触发点。

    用 cron + 容差判定,而非"今天的第一次推送":
    - 早报失败时,晚报不会错误升级为长版本
    - 容差直接绑定 cron 表达式,配置直观
    """
    morning_cron = config.get("schedule", {}).get("morning_cron")
    if not morning_cron:
        return False
    tolerance = timedelta(
        minutes=config["schedule"].get("morning_match_tolerance_minutes", 5)
    )
    base = now.replace(hour=0, minute=0, second=0, microsecond=0)
    today_fire = croniter(morning_cron, base).get_next(datetime)
    return abs(now - today_fire) <= tolerance
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_morning_detection.py -v`
Expected: 5 pass.

- [ ] **Step 5: Commit**

```bash
git add src/main.py tests/pytest/test_morning_detection.py
git commit -m "feat(main): add is_morning_push cron-based detector"
```

---

### Task 19: Add `_assemble_with_sentinels` helper

**Files:**
- Modify: `src/storage.py`
- Test: `tests/pytest/test_storage_sections.py`

- [ ] **Step 1: Write the failing test**

Append to `tests/pytest/test_storage_sections.py`:

```python
from storage import assemble_with_sentinels


class TestAssembleWithSentinels:
    def test_assembles_all_sections_in_order(self):
        out = assemble_with_sentinels(
            {"rss": "R", "github": "G", "hackernews": "H", "insights": "I"}
        )
        # 顺序:rss → github → hackernews → insights
        assert out.index("SECTION:rss") < out.index("SECTION:github")
        assert out.index("SECTION:github") < out.index("SECTION:hackernews")
        assert out.index("SECTION:hackernews") < out.index("SECTION:insights")
        assert "<!-- SECTION:rss BEGIN -->\nR\n<!-- SECTION:rss END -->" in out

    def test_omits_empty_sections(self):
        out = assemble_with_sentinels({"rss": "R", "github": "", "hackernews": "H", "insights": ""})
        assert "SECTION:github" not in out
        assert "SECTION:insights" not in out
        assert "SECTION:rss" in out
        assert "SECTION:hackernews" in out

    def test_returns_empty_when_all_empty(self):
        assert assemble_with_sentinels({"rss": "", "github": "", "hackernews": "", "insights": ""}) == ""
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_storage_sections.py::TestAssembleWithSentinels -v`
Expected: ImportError.

- [ ] **Step 3: Implement**

Append to `src/storage.py`:

```python
_SECTION_ORDER = ("rss", "github", "hackernews", "insights")


def assemble_with_sentinels(sections: Dict[str, str]) -> str:
    """按固定顺序拼装四段 markdown,每段包 sentinel;空段整段省略。"""
    parts: List[str] = []
    for key in _SECTION_ORDER:
        body = (sections.get(key) or "").strip()
        if not body:
            continue
        parts.append(f"<!-- SECTION:{key} BEGIN -->\n{body}\n<!-- SECTION:{key} END -->")
    return "\n\n".join(parts)
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_storage_sections.py::TestAssembleWithSentinels -v`
Expected: 3 pass.

- [ ] **Step 5: Commit**

```bash
git add src/storage.py tests/pytest/test_storage_sections.py
git commit -m "feat(storage): add assemble_with_sentinels helper"
```

---

### Task 20: Refactor `run_push_job` to dispatch by morning detection

**Files:**
- Modify: `src/main.py`
- Test: `tests/pytest/test_main_run_push_job.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_main_run_push_job.py`:

```python
"""测试 run_push_job 的早报/默认路径分发"""

import sys
from pathlib import Path
from unittest.mock import patch, AsyncMock

import pytest

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.main import run_push_job


@pytest.mark.asyncio
async def test_default_path_when_not_morning(sample_config):
    sample_config["schedule"]["morning_cron"] = "0 8 * * *"
    sample_config["schedule"]["morning_match_tolerance_minutes"] = 5
    sample_config["filter"]["push_context_days"] = 5

    with patch("src.main.is_morning_push", return_value=False), patch(
        "src.main._run_default_push", new=AsyncMock(return_value=None)
    ) as default_path, patch(
        "src.main._run_morning_push", new=AsyncMock(return_value=None)
    ) as morning_path:
        await run_push_job(sample_config)

    default_path.assert_awaited_once()
    morning_path.assert_not_awaited()


@pytest.mark.asyncio
async def test_morning_path_when_morning(sample_config):
    sample_config["schedule"]["morning_cron"] = "0 8 * * *"
    sample_config["schedule"]["morning_match_tolerance_minutes"] = 5
    sample_config["filter"]["push_context_days"] = 5

    with patch("src.main.is_morning_push", return_value=True), patch(
        "src.main._run_default_push", new=AsyncMock(return_value=None)
    ) as default_path, patch(
        "src.main._run_morning_push", new=AsyncMock(return_value=None)
    ) as morning_path:
        await run_push_job(sample_config)

    morning_path.assert_awaited_once()
    default_path.assert_not_awaited()
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_main_run_push_job.py -v`
Expected: AttributeError or ImportError on `_run_default_push` / `_run_morning_push`.

- [ ] **Step 3: Refactor `run_push_job`**

Open `src/main.py`. Replace the entire `async def run_push_job(config: Dict):` function (currently around lines 280-333) with:

```python
async def run_push_job(config: Dict):
    print(f"\n{'=' * 50}")
    print(f"📤 Push Job | {now_local().strftime('%Y-%m-%d %H:%M:%S')}")
    print(f"{'=' * 50}")

    if is_morning_push(now_local(config), config):
        await _run_morning_push(config)
    else:
        await _run_default_push(config)


async def _run_default_push(config: Dict):
    """晚报或非早报时段:沿用原有纯 RSS digest 流程"""
    last_push_file = get_last_push_file()
    last_push_time = extract_push_time(last_push_file) if last_push_file else None
    if last_push_time:
        print(f"📌 上次推送: {last_push_time.strftime('%Y-%m-%d %H:%M')}")

    min_score = config["filter"]["min_score"]
    context_days = config["filter"]["context_days"]
    to_push, context = collect_entries_for_push(
        last_push_time=last_push_time,
        context_days=context_days,
        min_score=min_score,
    )
    print(
        f"📋 待推送 {len(to_push)} / 上下文 {len(context)} (≥{min_score} 分)"
    )
    if not to_push:
        print("ℹ️ 没有新消息需要推送")
        return

    push_context_days = config["filter"].get("push_context_days", 5)
    recent = load_recent_push_titles(push_context_days)

    print("🤖 生成推送内容...")
    try:
        push_content = await compose_digest(
            to_push, context, config["llm"], recent_push_context=recent
        )
    except Exception as e:
        print(f"生成汇总推送失败: {e}")
        await notify_llm_errors("compose_digest", [str(e)], config)
        raise

    await send_to_platforms(push_content, config["push"])
    push_file = get_push_file()
    save_push_file(push_file, push_content, len(to_push), len(to_push), profile="default")
    print(f"💾 已保存到 {push_file}")
    print(f"✅ Push Job 完成 | 推送: {len(to_push)} 条")
```

(Note: `_run_morning_push` is implemented in the next task. For this task to typecheck, also add this stub now — Task 21 fills in the body.)

After `_run_default_push`, add:

```python
async def _run_morning_push(config: Dict):
    """早报时段:四模块编排 + sentinel 拼装 (Task 21 will implement)"""
    raise NotImplementedError("Task 21 implements morning push")
```

Also add the imports the new code needs at the top of main.py — find the existing `from src.storage import (` block and add `assemble_with_sentinels,` to it. Also add the section imports (will be used in Task 21):

```python
from src.sections.github.section import run_github_section
from src.sections.hackernews.section import run_hackernews_section
from src.sections.insights.section import run_insights_section
from src.sections.rss.section import run_rss_section
```

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_main_run_push_job.py -v`
Expected: 2 pass.

- [ ] **Step 5: Commit**

```bash
git add src/main.py tests/pytest/test_main_run_push_job.py
git commit -m "refactor(main): split run_push_job into default + morning dispatchers"
```

---

### Task 21: Implement `_run_morning_push` four-module orchestrator

**Files:**
- Modify: `src/main.py` (replace the stub)
- Test: `tests/pytest/test_main_morning_push.py`

- [ ] **Step 1: Write the failing test**

Create `tests/pytest/test_main_morning_push.py`:

```python
"""测试早报四模块编排:gather + insights 串行 + sentinel 拼装 + 失败隔离"""

import sys
from pathlib import Path
from unittest.mock import patch, AsyncMock

import pytest

sys.path.insert(0, str(Path(__file__).parent.parent.parent))

from src.main import _run_morning_push


@pytest.mark.asyncio
async def test_assembles_all_four_sections(sample_config):
    sample_config["filter"]["push_context_days"] = 5

    sent = {}
    async def fake_send(content, push_cfg):
        sent["content"] = content
    saved = {}
    def fake_save(filepath, content, source_count, total_entries, profile="default"):
        saved["profile"] = profile
        saved["content"] = content

    with patch("src.main.run_rss_section", new=AsyncMock(return_value=("R", None))), patch(
        "src.main.run_github_section", new=AsyncMock(return_value=("G", None))
    ), patch(
        "src.main.run_hackernews_section", new=AsyncMock(return_value=("H", None))
    ), patch(
        "src.main.run_insights_section", new=AsyncMock(return_value=("I", None))
    ), patch(
        "src.main.send_to_platforms", new=AsyncMock(side_effect=fake_send)
    ), patch(
        "src.main.save_push_file", side_effect=fake_save
    ):
        await _run_morning_push(sample_config)

    assert "SECTION:rss" in sent["content"]
    assert "SECTION:github" in sent["content"]
    assert "SECTION:hackernews" in sent["content"]
    assert "SECTION:insights" in sent["content"]
    assert saved["profile"] == "morning"


@pytest.mark.asyncio
async def test_rss_failure_raises_to_caller(sample_config):
    sample_config["filter"]["push_context_days"] = 5

    with patch(
        "src.main.run_rss_section", new=AsyncMock(return_value=("", "compose_digest 失败"))
    ), patch(
        "src.main.run_github_section", new=AsyncMock(return_value=("G", None))
    ), patch(
        "src.main.run_hackernews_section", new=AsyncMock(return_value=("H", None))
    ), patch(
        "src.main.notify_llm_errors", new=AsyncMock()
    ):
        with pytest.raises(RuntimeError):
            await _run_morning_push(sample_config)


@pytest.mark.asyncio
async def test_section_failure_degrades_to_omission(sample_config):
    sample_config["filter"]["push_context_days"] = 5

    sent = {}
    async def fake_send(content, push_cfg):
        sent["content"] = content

    with patch("src.main.run_rss_section", new=AsyncMock(return_value=("R", None))), patch(
        "src.main.run_github_section", new=AsyncMock(return_value=("", "gh down"))
    ), patch(
        "src.main.run_hackernews_section", new=AsyncMock(return_value=("H", None))
    ), patch(
        "src.main.run_insights_section", new=AsyncMock(return_value=("I", None))
    ), patch(
        "src.main.notify_llm_errors", new=AsyncMock()
    ), patch(
        "src.main.send_to_platforms", new=AsyncMock(side_effect=fake_send)
    ), patch(
        "src.main.save_push_file"
    ):
        await _run_morning_push(sample_config)

    assert "SECTION:rss" in sent["content"]
    assert "SECTION:github" not in sent["content"]
    assert "SECTION:hackernews" in sent["content"]
```

- [ ] **Step 2: Run test to verify it fails**

Run: `uv run pytest tests/pytest/test_main_morning_push.py -v`
Expected: NotImplementedError (from the stub).

- [ ] **Step 3: Implement `_run_morning_push`**

In `src/main.py`, replace the stub:

```python
async def _run_morning_push(config: Dict):
    """早报四模块编排:RSS/GH/HN 并发 → insights 串行 → sentinel 拼装 → 推送 → 落盘。

    失败语义:
    - RSS 失败 → 整体抛 RuntimeError (核心承诺不变)
    - GH/HN/insights 失败 → 该段省略 + 告警,其他段照推
    """
    now = now_local(config)

    rss_result, gh_result, hn_result = await asyncio.gather(
        run_rss_section(config, now),
        run_github_section(config, now),
        run_hackernews_section(config, now),
    )

    rss_md, rss_err = rss_result
    gh_md, gh_err = gh_result
    hn_md, hn_err = hn_result

    # 失败告警(GH/HN 非阻塞)
    if gh_err:
        await notify_llm_errors("section_github", [gh_err], config)
    if hn_err:
        await notify_llm_errors("section_hackernews", [hn_err], config)

    # RSS 失败阻断
    if rss_err and not rss_md:
        await notify_llm_errors("compose_digest", [rss_err], config)
        raise RuntimeError(f"RSS section failed: {rss_err}")

    # insights(串行)
    insights_md, insights_err = await run_insights_section(
        rss_md, gh_md, hn_md, config, now
    )
    if insights_err:
        await notify_llm_errors("insights", [insights_err], config)

    final = assemble_with_sentinels(
        {
            "rss": rss_md,
            "github": gh_md,
            "hackernews": hn_md,
            "insights": insights_md,
        }
    )

    if not final.strip():
        print("ℹ️ 早报无任何段输出,跳过推送")
        return

    await send_to_platforms(final, config["push"])
    push_file = get_push_file()
    # source/total 在早报场景下意义弱化;沿用 RSS 段长度作为弱代理
    rss_count = rss_md.count("###") if rss_md else 0
    save_push_file(push_file, final, rss_count, rss_count, profile="morning")
    print(f"💾 已保存早报到 {push_file}")
```

Also ensure `asyncio` is imported at the top of `src/main.py` (it already is — line 4).

- [ ] **Step 4: Run test to verify it passes**

Run: `uv run pytest tests/pytest/test_main_morning_push.py tests/pytest/test_main_run_push_job.py -v`
Expected: all pass.

- [ ] **Step 5: Run all tests for regression check**

Run: `uv run pytest tests/pytest/ -v`
Expected: all pass.

- [ ] **Step 6: Commit**

```bash
git add src/main.py tests/pytest/test_main_morning_push.py
git commit -m "feat(main): implement four-module morning push orchestrator"
```

---

## Phase 7: Config, Docs, Integration Scripts

### Task 22: Update `config.json` and `config.json.example`

**Files:**
- Modify: `config.json`
- Modify: `config.json.example`

- [ ] **Step 1: Update `config.json`**

Open `config.json` and (a) add `"morning_cron"` and `"morning_match_tolerance_minutes"` under `schedule`, (b) add the entire `sections` block, (c) add the four new prompt paths under `llm.prompts`.

Insert under `schedule` (after `"push_cron"`):

```json
        "morning_cron": "0 8 * * *",
        "morning_match_tolerance_minutes": 5,
```

After the `schedule` block (or anywhere at top level), add:

```json
    "sections": {
        "github_trending": {
            "enabled": true,
            "max_items": 3,
            "max_deep_dive": 10,
            "readme_max_chars": 3000,
            "history_file": "news-data/trending-history.json",
            "request_timeout": 10,
            "tokenName": "GITHUB_TOKEN"
        },
        "hackernews": {
            "enabled": true,
            "select_k": 1,
            "top_comments": 20,
            "comment_max_chars": 500,
            "link_content_max_chars": 3000,
            "request_timeout": 10,
            "algolia_base": "https://hn.algolia.com/api/v1"
        },
        "insights": {
            "enabled": true
        }
    },
```

In the `llm.prompts` block, add:

```json
            "section_github": "prompts/section_github.md",
            "section_hackernews_select": "prompts/section_hackernews_select.md",
            "section_hackernews": "prompts/section_hackernews.md",
            "insights": "prompts/insights.md"
```

- [ ] **Step 2: Mirror the changes into `config.json.example`**

Apply the same edits to `config.json.example`.

- [ ] **Step 3: Validate JSON**

Run: `uv run python -c "import json; json.load(open('config.json')); json.load(open('config.json.example')); print('ok')"`
Expected: `ok`

- [ ] **Step 4: Confirm config loads cleanly**

Run: `uv run python -c "from src.config import load_config; c = load_config(); print(c['sections']['github_trending']['enabled'])"`
Expected: `True`

- [ ] **Step 5: Commit**

```bash
git add config.json config.json.example
git commit -m "feat(config): add sections + morning_cron schema"
```

---

### Task 23: Update `docs/tech-spec.md` (architecture sync)

**Files:**
- Modify: `docs/tech-spec.md`

- [ ] **Step 1: Add 板块编排 section**

Find the "## 关键模块边界" section in `docs/tech-spec.md`. After the existing `src/` tree block, add:

```markdown

### 板块化扩展 (morning push)

早报推送在 RSS 之上扩展三个板块:GitHub 趋势 / Hacker News 热议 / 跨板块洞察。模块结构、数据流与失败降级详见 `docs/extra-sections-design.md`。架构层关键约束:

- 仅在 `schedule.morning_cron` 命中(± `morning_match_tolerance_minutes`)时启用,晚报维持纯 RSS 行为
- 各板块封装为 `src/sections/<board>/section.py::run_xxx_section(config, now) -> (markdown, error)`
- `push_job` 用 `asyncio.gather` 并发跑 RSS / GH / HN,串行接 insights;最后用 `<!-- SECTION:xxx BEGIN/END -->` sentinel 包入 push 文件
- 仅 RSS 失败会让 push_job 整体退出非 0;其余板块失败 → 板块整段省略 + 告警

新增持久化文件:`news-data/trending-history.json`(GH 已查阅 repo 索引,按 `filter.keep_days` 过期)
```

- [ ] **Step 2: Update the data flow diagram (optional touch)**

In the same file's "### 数据流" section, leave the existing mermaid flowchart unchanged (it describes the RSS path); the new design doc carries the morning-specific flow.

- [ ] **Step 3: Update the maintenance date**

Find `> update: 2026-05-15` near the top and change to `> update: 2026-05-17`.

- [ ] **Step 4: Commit**

```bash
git add docs/tech-spec.md
git commit -m "docs(tech-spec): sync architecture with morning extra sections"
```

---

### Task 24: Update `README.md` config detail section

**Files:**
- Modify: `README.md`

- [ ] **Step 1: Locate config detail section**

Open `README.md`. Find the section that documents `config.json` fields (typically headed "配置详解" or similar). Identify where `schedule` and `llm.prompts` are documented.

- [ ] **Step 2: Add `schedule.morning_cron` description**

Under `schedule` documentation, add:

```markdown
- `morning_cron` (可选): 早报 cron 表达式 (例: `"0 8 * * *"`)。命中时启用三个额外板块(GitHub / HN / 洞察)。缺失则全程纯 RSS。
- `morning_match_tolerance_minutes` (可选,默认 5): 早报判定容差(分钟)。
```

- [ ] **Step 3: Add `sections` block documentation**

Add a new subsection:

```markdown
### sections (早报扩展板块)

仅在 `schedule.morning_cron` 命中时生效。详细设计见 `docs/extra-sections-design.md`。

#### sections.github_trending
- `enabled`: 是否启用
- `max_items`: LLM 最终选出的项目数上限(默认 3)
- `max_deep_dive`: 单次最多 deep-dive 的候选 repo 数(默认 10)
- `readme_max_chars`: README 截断长度(默认 3000)
- `history_file`: trending 去重索引文件路径
- `request_timeout`: HTTP 超时秒
- `tokenName`: GitHub token 环境变量名;不设时匿名调用(限 60 req/hr)

#### sections.hackernews
- `enabled`: 是否启用
- `select_k`: 轻 LLM 从首页 30 条中挑出的故事数(默认 1)
- `top_comments`: 每个故事抓取的顶层评论数(默认 20)
- `comment_max_chars`: 单条评论截断长度(默认 500)
- `link_content_max_chars`: 外链正文截断长度(默认 3000)
- `request_timeout`: HTTP 超时秒
- `algolia_base`: Algolia API 基址

#### sections.insights
- `enabled`: 是否启用跨板块洞察段
```

- [ ] **Step 4: Add GITHUB_TOKEN to env vars table**

In the env vars section, add:

```markdown
- `GITHUB_TOKEN` (可选): GitHub API token,提升 deep-dive 限额到 5000 req/hr。不设时匿名调用(60 req/hr,日 20 calls 量级安全)。
```

- [ ] **Step 5: Commit**

```bash
git add README.md
git commit -m "docs(readme): document morning sections config + GITHUB_TOKEN"
```

---

### Task 25: Update `docs/plan.md`

**Files:**
- Modify: `docs/plan.md`

- [ ] **Step 1: Append decision + progress entries**

Open `docs/plan.md`. Find the `## 技术决策记录` heading. Append:

```markdown
### 2026-05-17: 早报扩展板块

- 决策:在 RSS digest 之上为早报增加 GitHub Trending / Hacker News / 跨板块洞察三段
- 触发条件:`schedule.morning_cron` 命中(± `morning_match_tolerance_minutes`)
- 模块边界:四个自治模块在 `src/sections/<board>/`,push_job 上游统一包 sentinel
- GH:单页 trending HTML 抓取 → history 去重 → REST API 拿 README+topics+metadata → LLM 选 1-3
- HN:首页 HTML → 轻 LLM 选 K(默认 1)→ Algolia API 拉评论 + html_to_markdown 拉外链 → LLM 行文
- 失败语义:RSS 失败整体退出;其余板块失败省略本段 + 告警
- 详细设计:`docs/extra-sections-design.md`;实施计划:`docs/superpowers/plans/2026-05-17-extra-sections.md`
```

Find `## 开发进度` heading. Append:

```markdown
### 2026-05-17

- ✅ 设计完成,详见 `docs/extra-sections-design.md`
- ✅ 实施计划完成,详见 `docs/superpowers/plans/2026-05-17-extra-sections.md`
- 🔄 实施中(按计划分 25 个任务推进)
```

- [ ] **Step 2: Commit**

```bash
git add docs/plan.md
git commit -m "docs(plan): record morning-sections decisions + progress"
```

---

### Task 26: Add manual test scripts

**Files:**
- Create: `tests/fetch_trending.py`
- Create: `tests/fetch_hackernews.py`
- Create: `tests/run_morning_push.py`

- [ ] **Step 1: Create `tests/fetch_trending.py`**

```python
"""手动跑一次 GH trending 抓取 + deep-dive,验证选择器与 API 接入"""

import asyncio
import json
import os
import sys

from dotenv import load_dotenv

load_dotenv()
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

from src.config import load_config
from src.sections.github.repo_enricher import enrich_repos
from src.sections.github.trending_scraper import (
    fetch_trending_page,
    parse_trending_html,
)


async def main():
    config = load_config()
    print("📥 抓取 GitHub Trending...")
    html = await fetch_trending_page(timeout=15)
    repos = parse_trending_html(html)
    print(f"📋 解析出 {len(repos)} 个 repo")
    for r in repos[:5]:
        print(f"  - {r['full_name']} ⭐{r['stars_today']}/{r['stars_total']} | {r['description'][:80]}")

    cfg = config["sections"]["github_trending"]
    print(f"\n🔍 enrich 前 {min(3, len(repos))} 个...")
    enriched, errors = await enrich_repos(
        repos[:3],
        token_env=cfg.get("tokenName", "GITHUB_TOKEN"),
        readme_max_chars=cfg.get("readme_max_chars", 3000),
        timeout=15,
    )
    for e in errors:
        print(f"  ⚠️ {e}")
    print(json.dumps(enriched, ensure_ascii=False, indent=2))


if __name__ == "__main__":
    asyncio.run(main())
```

- [ ] **Step 2: Create `tests/fetch_hackernews.py`**

```python
"""手动跑一次 HN 首页 + Algolia enrich,验证选择器与 API"""

import asyncio
import json
import os
import sys

from dotenv import load_dotenv

load_dotenv()
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

from src.sections.hackernews.frontpage_scraper import (
    fetch_frontpage,
    parse_frontpage_html,
)
from src.sections.hackernews.item_enricher import enrich_stories


async def main():
    print("📥 抓取 HN 首页...")
    html = await fetch_frontpage(timeout=15)
    stories = parse_frontpage_html(html)
    print(f"📋 解析出 {len(stories)} 条")
    for s in stories[:5]:
        print(f"  - [{s['points']} pts · {s['comments']} comments] {s['title']} ({s['site']})")

    print("\n🔍 enrich 前 1 个外链类故事...")
    target = next((s for s in stories if not s["url"].startswith("https://news.ycombinator.com/")), stories[0])
    enriched, errors = await enrich_stories(
        [target],
        top_comments=5,
        comment_max_chars=300,
        link_content_max_chars=1500,
        timeout=15,
    )
    for e in errors:
        print(f"  ⚠️ {e}")
    print(json.dumps(enriched, ensure_ascii=False, indent=2))


if __name__ == "__main__":
    asyncio.run(main())
```

- [ ] **Step 3: Create `tests/run_morning_push.py`**

```python
"""模拟一次完整早报推送(强制 is_morning=True,但不发送到推送渠道)"""

import asyncio
import os
import sys
from unittest.mock import AsyncMock, patch

from dotenv import load_dotenv

load_dotenv()
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

from src.config import load_config
from src.main import _run_morning_push


async def main():
    config = load_config()
    # 拦截真实推送,改为打印
    async def fake_send(content, push_cfg):
        print("\n" + "=" * 60)
        print("📤 假推送内容(实际不会发送)")
        print("=" * 60)
        print(content)

    with patch("src.main.send_to_platforms", new=AsyncMock(side_effect=fake_send)):
        await _run_morning_push(config)


if __name__ == "__main__":
    asyncio.run(main())
```

- [ ] **Step 4: Verify scripts at least parse**

Run: `uv run python -c "import ast; [ast.parse(open(f).read()) for f in ['tests/fetch_trending.py', 'tests/fetch_hackernews.py', 'tests/run_morning_push.py']]; print('ok')"`
Expected: `ok`

- [ ] **Step 5: Commit**

```bash
git add tests/fetch_trending.py tests/fetch_hackernews.py tests/run_morning_push.py
git commit -m "test: add interactive scripts for trending/HN/morning push smoke tests"
```

---

### Task 27: End-to-end smoke test with real APIs

This is a manual verification step before declaring the feature done.

- [ ] **Step 1: Run GH trending smoke test**

Run: `uv run python tests/fetch_trending.py`
Expected: prints 25+ repos, enriches 3, shows topics/readme_excerpt in JSON output.

- [ ] **Step 2: Run HN smoke test**

Run: `uv run python tests/fetch_hackernews.py`
Expected: prints 30 stories with points/comments, enriches 1 with link_content + top_comments.

- [ ] **Step 3: Run full morning push simulation**

Run: `uv run python tests/run_morning_push.py`
Expected: terminal prints the assembled push markdown containing 4 sections (or fewer with sentinel-bound omissions for any failing section). RSS section must be present (else the run aborts).

- [ ] **Step 4: Run full pytest suite**

Run: `uv run pytest tests/pytest/ -v`
Expected: all pass (including pre-existing tests).

- [ ] **Step 5: Commit nothing, but update plan.md progress**

Open `docs/plan.md` and update the most recent entry under `### 2026-05-17` from `🔄 实施中` to `✅ 实施完成`.

```bash
git add docs/plan.md
git commit -m "docs(plan): mark morning-sections implementation complete"
```

---

## Self-Review

**Spec coverage:**
- §2 architecture (4 modules + push_job orchestration) → Tasks 6, 10, 15, 17, 20, 21 ✓
- §3 module structure (`src/sections/...`) → created across Tasks 6, 8, 10, 13, 15, 17 ✓
- §4.1 sentinel contract → Task 19 (`assemble_with_sentinels`) ✓
- §4.2 `extract_section` + `load_recent_section_titles` → Tasks 1, 2 ✓
- §4.3 trending-history → Tasks 3, 5 ✓
- §5.1 RSS migration → Task 6 ✓
- §5.2 GitHub module flow → Tasks 7-11 ✓
- §5.3 Hacker News module flow → Tasks 12-16 ✓
- §5.4 Insights module → Task 17 ✓
- §6.1 LLM functions → Tasks 11, 16, 17 ✓
- §6.2 关注领域 + §6.3 prompts → Tasks 11, 16, 17 (prompts created with focus domain inline) ✓
- §6.4 调用顺序 → Task 21 ✓
- §7 insights formatting deferred to prompt → Task 17 prompt body honors this ✓
- §8 config schema → Task 22 ✓
- §9 failure isolation → Tasks 10, 15, 17, 21 ✓
- §10 morning detection → Task 18 ✓
- §11 integration points → Tasks 4 (save_push_file profile), 5 (cleanup history), 20 (main.py refactor) ✓
- §12 test strategy → Tasks 1-21 (unit), Task 26 (interactive scripts) ✓
- §13 implementation steps → matched 1:1 ✓

**Placeholder scan:** No TBDs; every code step has runnable code. Manual smoke tests in Task 27 are unavoidable manual verifications, called out explicitly.

**Type consistency:**
- `TrendingHistory.touch(url, today)` — signature is consistent across Task 3, Task 5, Task 10.
- `run_<board>_section(config, now) -> (str, Optional[str])` — consistent across Tasks 6, 10, 15.
- `run_insights_section(rss_md, gh_md, hn_md, config, now) -> (str, Optional[str])` — consistent across Task 17 + Task 21.
- `assemble_with_sentinels(dict[str, str]) -> str` — consistent in Task 19 definition + Task 21 use.
- `save_push_file(filepath, content, source_count, total_entries, profile="default")` — consistent in Task 4 definition + Task 20 default-path call + Task 21 morning-path call.

No issues found.