Release Watchers — Process Overview
Reference doc for the automated doc-need detection built into this repo. Explains the whole flow, both workflows, what each step actually does, and where the prompts/code live. Covers only the automated front half of the pipeline (detecting a doc need) — pre/post-release checklist verification is a separate, manually-triggered process, kept out of this branch entirely.Overview — the problem this solves
Before this existed, the gap between “code merged upstream” and “someone realizes a doc is needed” was entirely manual, and mostly only caught around release time. Everything after a doc PR already exists was already automated (style review, broken-link checks); nothing watched for the need to exist in the first place. Two independent, symmetric workflows close that gap — one per release type, because they’re genuinely different problems with different cadences and different risks, not one problem with a mode switch./create-draft), then the same review automation everything else in this
repo already goes through.
Design principles
- Deterministic gates, never an AI judgment call for “should this run today” or “what version is current.” Cadence (which days, which lookback window) and the current/next version numbers are all decided by plain bash before Claude ever runs — the AI’s job is judging PR content, not deciding whether today is a scan day or what version comes next.
- Self-healing lookback, not a fixed day-count. Both workflows ask
“when did I last succeed?” via
gh run list(filtered to realscheduleruns, so a manual dry-run test can never contaminate the baseline), and use that as the actual cutoff. A missed or failed run widens the next lookback automatically instead of silently dropping PRs. Asinceworkflow_dispatch input can override this outright for a one-off catch-up run — needed the first time each workflow runs for real, when there’s no prior run to compute a baseline from. - Current version comes from this repo’s own published release notes,
not a separately-maintained config file.
snippets/releases/latest.mdxis replaced wholesale on every real release — it’s the one thing that’s already guaranteed to reflect an actual publish decision a human made, so both workflows parse the version straight out of it and compute next-minor/next-major from that, deterministically, in bash. - Verify before notify. Every candidate is classified as CONFIRMED, RULED OUT, NEEDS A LOOK, or HELD before anything reaches an issue — a passing title is never taken at face value.
- Dry-run is a real feature, not a testing hack. Both workflows accept
a
dry_runinput that runs every read-only step for real but never writes anything — useful for previewing before trusting a new filter tweak, not just something built for this one round of testing. A workflow-levelFORCE_DRY_RUNswitch additionally forces this on regardless of trigger (schedule included), by removing the write tools from Claude’s allowed list outright — see “First live rollout” below. Because no issue-write tools exist in that mode, there’s no digest issue to read afterward — the notify job instead writes what it would have created or updated todry-run-preview.md, which a deterministic step publishes to the workflow run’s job summary. - No guessing on ambiguity, ever. Major-vs-minor classification, doc-relevance, and feature completeness all have an explicit “I don’t know” outcome that surfaces the evidence to a human instead of forcing a binary call.
Workflow 1 — Minor Release Watcher
File:.github/workflows/minor-release-watcher.yml
Schedule: 0 6 * * 1,4 — 06:00 UTC, every Monday and Thursday
Scope: the single current minor/patch line only (e.g.
2.0.0 → 2.0.1 → 2.0.2). Never touches main — that’s the other
workflow’s job, and a PR merged straight to a release branch is
unambiguously minor-bound, so there’s no classification judgment needed on
this side. Older lines (e.g. a still-patched 1.13.x) are not watched —
a deliberate scope decision, not an oversight.
Workflow 2 — Major Release Watcher
File:.github/workflows/major-release-watcher.yml
Schedule: 0 6 * * 1 — every Monday, but a deterministic ISO-week-parity
check (date -u +%V, even weeks only) means it only actually does anything
every other Monday. workflow_dispatch accepts a force_run input to
override this for a manual run.
Scope: main only, targeting the next major version (computed
deterministically from the current version — e.g. current 2.0 → next
major 2.1). Does two jobs in one run:
Part A — scan for new major-bound doc needs
Part B — completeness check on already-tracked features
Part C / D — Digest and Slack
Same shape as the minor watcher’s steps 7–8, but the digest issue is titled “Major Release Watcher — Scan Digest” and covers both Part A and Part B in one comment per run.First live rollout
Both workflows carry a workflow-levelenv: FORCE_DRY_RUN: "true". While
set, this overrides dry_run regardless of trigger (schedule included),
and Claude’s allowed tools have the write commands (gh issue create/edit/comment) removed outright — not just described as
off-limits in the prompt. This makes the first pass after merging safe by
construction: the schedule can fire for real before anyone tests it
manually, and that run can only ever read and report.
The intended sequence once merged:
- Run each workflow via
workflow_dispatchfrommain. - Open that run’s job summary in the Actions UI and sanity-check the dry-run preview against what’s actually open upstream — forced dry-run creates no digest issue to read instead, since the tools to create one aren’t available in that mode.
- Flip
FORCE_DRY_RUNto"false"in its own separate, reviewable commit — that’s the actual go-live moment.
Schedule summary
Secrets and variables this depends on
A Projects-board integration (org project #105, “Documentation”) was
scoped during design but never wired in — still needs a token with
Projects (v2) write access if that’s wanted later.
Known limitations / what’s still open
- No project-board integration yet — issues are tracked as plain GitHub issues with assignees, not added to project #105.
- Email notification was scoped, not built — parked pending a decision on sending method (SMTP vs. a transactional API) and frequency.
- Old pre-split tracked issues (#428, #450, #452) use the original
daily-watcher:pr-Xmarker. Dedup searches by number rather than prefix specifically to keep recognizing these, but they were never retroactively migrated to carry aminor-watcher/major-watchermarker of their own. - First run for each workflow has no history to look back on — the
sinceinput handles this deliberately; without it, falls back to a fixed generous window (4 days for minor, 15 for major) rather than guessing. - Older minor lines (e.g.
1.13.x) are no longer watched — a deliberate scope reduction to a single current line, not a bug. create-draft.yml’s real git-push/PR-open path has never executed successfully —issue_commentonly fires from the default branch, so this can only be tested for real after merging.
Verified before this went anywhere near main
Everything below was checked on disposable branches (never a PR) —
see the commit history on docs/daily-watcher-docs-om for full detail.
Being specific here on purpose: an earlier version of this doc overstated
what had actually been confirmed.
Confirmed working, via real live-API calls:
- The plain
ghCLI commands both workflows depend on (label create, issue create/comment/edit/close) work against the live API - The lookback mechanism: an earlier repo-variable approach was
confirmed broken (the default
GITHUB_TOKENgets a 403 regardless of declared permissions) before being replaced withgh run list --event schedule, itself confirmed to filter correctly by trigger event - The major watcher’s off-week-run detection (Jobs API step-conclusion check, not just run status)
- The
sinceoverride and theFORCE_DRY_RUNsafety switch (gating logic, tool-list construction, andDRY_RUNresolution all confirmed on disposable branches) - The version-derivation step: confirmed the parsed version, computed next-minor/next-major/branch, and the two derived directory names all match real data and real directories on disk
- De-duplication against both marker prefixes and the “Backport #X” title pattern, verified against real upstream PRs
- Major/minor classification logic, walked against real PRs both ways
schedule, workflow_dispatch, and issue_comment only fire from a
repo’s default branch. The actual claude-code-action step (the part
that reads real PRs and would create/update issues) has never executed
successfully in a real run yet — only the surrounding bash/gh logic
above has been proven live. See “First live rollout” above for how this
is being handled safely.