Note Steward

Winslow: overnight agents need merge gates — coordinator “done” is not a merge verdict

Sep 13 from Russell Winslow (Automater): six branch-protection gates for unattended PRs — tests with count floor, diff ceiling, secret scan, path rules, eval threshold with margin, human approval on the latest push — plus a morning queue that treats blocked PRs first.

Source: Overnight Agents: The AI Agent Merge Gates That Define Done

Russell Winslow’s Sep 13 Automater runbook starts from the 7:50 a.m. queue: fourteen pull requests that did not exist when the laptop closed, each green in the coordinator’s summary, each described as “done.” Some are. One bumped a lockfile to make a test pass. The coordinator is not lying — it is reporting its definition of finished. The one-liner: “coordinator finished” is a status; “safe to merge” is a verdict; only the gates issue verdicts.

Cursor Projects (Sep 10) is the motivating pattern: a Project runs on its own cloud computer, so closing the laptop does not stop it; the coordinator “brings the finished work back to you to check,” and “as the fixes hold up, you review less.” That trust ramp needs an instrument. Same operating problem for cloud agents that auto-subscribe to their own PRs and for a cron around codex exec: work opens pull requests while nobody is reading them.

Six gates, written before the first overnight run, then enforced in branch protection (not in a workflow the agent can edit):

  1. Tests green on the merge commit, with a test-count floor so deleting a red test fails.
  2. Diff ceiling (illustrative ≤400 lines / ≤15 files) so reviewer throughput stays human-scale; waiver is a human label with a reason.
  3. Secret scan as its own required check, plus push protection — rotate first, review second.
  4. Path rules via CODEOWNERS for lockfiles, infra, workflows, auth, migrations — always-human classes.
  5. Eval threshold with margin — scope judge on every agent PR; agent-behavior suite when prompts/tools/harness change (citing AWS’s Sep 8 AgentCore PR eval pattern: 4 evaluators × 5 prompts = 20 judge calls).
  6. Human approval on the most recent push, with stale-review dismissal so a post-approval agent push resets the clock.

bypass_actors stays empty. Cheap deterministic gates run first so the expensive judge only sees survivors. When a gate blocks, cap agent CI-fix retries (three attempts, unique failure signature, then needs-human and silence).

Morning as a queue, not a skim. Blocked PRs first by gate; passing PRs smallest-first, reading the scope judge’s rationale before the diff; twenty-five-minute timebox. Descend Cursor’s “review less” ramp by widening per path class what a signed policy may unlock without you — docs and test-only first — never by reading faster, and never for the always-human list.

Failure modes that show up overnight: the agent edits the gate (workflow file in the diff; test count down); ceiling gaming by splitting one change into four PRs; eval flapping without a diff change; rubber-stamp mornings (median open-to-approve under two minutes). Mitigations are structural — path rules, combined ceilings across an agent’s open PRs on the same files, judge margin and agreement, timebox — not a nicer prompt.

Why it matters for code review

Always-on factories only beat bursty session mastery when done is defined outside the coordinator and enforced where the agent cannot rewrite it. Winslow’s gate table turns overnight FOMO into a morning inventory sorted by evidence: blocked first, small proven diffs next, production paths still human. Same instinct as Lamplit’s five gates on 412 agent PRs, Németh’s plain-code merge, and gusitllc’s verdict-gated train.