# Reopen-on-feedback kit

Make sure a human correction on a published-track claim re-verifies everything from primary
sources, instead of patching the one flagged line and keeping the rest of an old verdict.

Full write-up and the real incident that produced this kit:
https://kittyclaw.dev/the-fact-check-that-made-a-true-claim-false

## The trap this closes

A primary source (a license file, a settings page, a commit) can already be up to date while a
secondary index of it (a cached badge, a directory listing, a search snippet) still shows the
old value. If a verification pass resolves a claim against that lagging index instead of
opening the primary source itself, it can "correct" an already-true claim into a false one -
even though nothing about the underlying fact changed during the check. That is the exact
failure this kit is built from: a draft claim was accurate because the primary source had
already changed days earlier, and the first verification pass flipped that true claim to
false - most likely by following a secondary index that hadn't caught up yet, though which
exact source it consulted was never logged, so that mechanism is the likely explanation, not
a proven one. When a human then points out the
error, the easy move is to patch just that one sentence and leave every other claim's old
"verified" status standing. That's the trap: the old verdict didn't get less true because one
claim turned out wrong, but it also didn't get re-checked. This kit forces a full re-run
instead, resolved against primary sources only.

## The two roles

### 1. Editor

- Applies the correction described in the human's comment.
- Never edits the claim in place and resubmits directly to a "verified" or "done" state -
  always routes back through the same chain the first draft went through (edit, then
  re-confirm any attached assets, then fact-check), so nothing downstream of the fix is
  silently assumed unchanged.

### 2. Fact-checker

- Triggered automatically on **every** entry to the fact-check column - including a
  re-entry after a correction (see `automation.json`). There is deliberately no condition
  that skips the run because a PASS comment already exists on the ticket; that old comment
  belongs to a state that no longer exists.
- Re-derives every claim from its primary source again, from scratch. It does not read its
  own previous comment on the same ticket as a starting point.
- For any claim about your own product (license, version, price, capacity limits): resolve
  against the primary source of truth (the actual file, the actual settings, the actual
  commit) - never a secondary index of it (a README badge, a marketplace listing, a cached
  docs page, a search result). Secondary indexes lag the thing they describe; that lag is
  exactly what causes a "verified" claim to go stale without anyone changing the claim
  itself.
- Posts a fresh report with a fresh **PASS**/**FAIL** verdict, dated. Never carries a
  previous run's status forward.

## Columns and routes

```
Draft → Edit → (assets) → FactCheck (fresh run, every time) → Review
                                                                  │
                                                    human correction on a claim
                                                                  │
                                                                  ▼
                                              Edit (apply the fix) → (assets, re-confirmed)
                                                    → FactCheck (fresh run, ignores the old PASS)
                                                                  │
                                                                  ▼
                                                               Review → Publish
```

The important property is the same one as in the publishing-guardrail kit: what's **missing**
matters more than what's there. `automation.json` wires the fact-check trigger to the column
itself (`to: FactCheck`), not to "first time only" or "unless already verified." A ticket
re-entering that column after a correction gets the exact same treatment as a ticket entering
it for the first time.

## Report format

```
## Re-check report

Triggered by: [initial draft / human correction]
Previous verdict on this ticket: [ignored by design - not read before this run]

### Claims re-verified
- Total: X (Y OK, Z fixed this pass)

### Verdict
PASS / FAIL
```

## Install

1. Copy `automation.json`'s rule into your own automation/rule config, adjusting the column
   name and agent slug for your fact-checker role.
2. Do **not** add a condition that skips the run when the ticket already carries a passing
   verdict comment. That condition is the exact bug this kit exists to prevent.
3. Add "Never resubmit a corrected claim directly to a verified state - route back through
   the same edit → fact-check chain the original draft used" to your editor role's
   instructions.
4. Add "Re-derive every claim from its primary source on every run; never read your own
   previous comment on this ticket as a shortcut" to your fact-checker role's instructions.

## Test the gate

**Status: this walkthrough is a written procedure for you to run on your own board, not a
run we have already executed and captured end to end on our side.** `automation.json`'s JSON
is valid and its trigger/action shape matches a real automation already running on our board
(see the publishing-guardrail kit), but we have not provisioned a dedicated test project and
attached reports proving the re-entry-to-FactCheck trigger fires exactly as described. Treat
steps 1-7 below as instructions to verify the gate yourself before relying on it, at the same
level of proof as the other kits on this site.

Use `sample-ticket.md` - a fully synthetic example with no private data, built to reproduce the
exact failure mode above (a primary source already correct, a lagging secondary index, a first
pass that trusts the index and turns a true claim false):

1. Create the ticket as written: the draft claim is already accurate, because the synthetic
   "primary source" it's checked against already reflects the current value.
2. Run the fact-checker role once and confirm it reproduces the bug on purpose: with the
   lagging "secondary index" resource included in the ticket, the first pass should resolve
   the claim against that index and post an incorrect **PASS** on the wrong value - proving
   the trap is real before you test the fix for it.
3. Post the synthetic human correction as a new comment and move the ticket back to Edit.
4. Apply the one-line fix, then move the ticket back to FactCheck.
5. Confirm the automation fires the fact-checker role again - not a shortcut that only
   reads the new comment - and that its report re-derives every claim from the primary
   source this time, not the lagging index, and re-verifies every claim on the ticket, not
   only the corrected one.
6. Confirm the new report carries today's date and a fresh verdict, and does not reference
   or inherit the earlier (incorrect) PASS as still valid.