Review Pipeline

Core Rule

Parallelize independent auditors over a PR-scope diff. Dedupe across reports. Never widen scope beyond the diff.

Kit Context

Before starting this skill, ensure you have completed session boot:

1. Read CODEBASE_MAP.md for project understanding
2. Read CLAUDE.project.md if it exists for project-specific rules
3. Read tasks/lessons/_index.md for accumulated corrections (Top Rules + index)

If any of these haven't been read in this session, read them now before proceeding.

When to Use

Invoke with /review-pipeline when:

• A PR is ready and you want a multi-lens check before requesting human review
• A multi-session feature has landed on a branch and you want to catch regressions
• You're investigating a flaky area and want several auditors to weigh in
• You suspect issues but don't know which audit applies — let the pipeline pick

Not for:

• Trivial single-file edits — invoke the relevant audit directly
• Project-wide periodic health checks — use /project-health-report instead
• Full architectural assessments — use /architecture-review or /deepening-review

Scope Rules

• Read-only — produces a report, never modifies code
• Operates within the detected scope (recent changes by default)
• Logs unrelated issues found during analysis under tasks/todo.md > ## Not Now
• Does not duplicate findings already in CLAUDE.md or tasks/lessons/

Process

Phase 1: Resolve Scope

Pick the smallest scope that captures the change:

1. If the user passed paths or globs — use those literally
2. Else if the working tree has uncommitted changes — use git diff --name-only HEAD
3. Else if on a feature branch — use git diff --name-only $(git merge-base HEAD origin/main)...HEAD (fall back to main if origin/main is absent)
4. Else — ask the user which path or commit range to review; do not silently default to the whole repo

Filter the file list:

• Drop generated paths (node_modules/, dist/, build/, .next/, __pycache__/, vendor/)
• Drop lockfiles unless dependency-audit is selected
• If the resulting list is empty, stop and report "no in-scope changes detected"

Save the resolved scope as a bulleted list — every parallel auditor gets the same list.

Phase 2: Select Audits

Always select these core audits:

• code-quality-audit — smells, error handling, maintainability
• testing-audit — coverage gaps, test quality

Conditionally add based on file patterns in the resolved scope:

Cap at 5 parallel audits by default to keep token spend bounded. If more are eligible, pick the 5 with the strongest fit to the scope and list the skipped ones in the report's "Skipped audits" section with a one-line rationale.

If the user passed an explicit list (e.g., /review-pipeline testing,security), honor it literally and skip selection logic.

Phase 3: Run Audits in Parallel

For each selected audit, dispatch one Task in a single message containing all tool calls so they run concurrently. Pattern per Task:

Task(
  description: "<audit-name> on <short scope label>",
  subagent_type: "general-purpose",
  prompt: """
    You are running the <audit-name> audit on a scoped set of files.

    Step 1 — Read the skill file at `.claude/skills/<audit-name>/SKILL.md` and follow
    its Process section. Do not invoke other audits; do not analyze files outside
    the scope below.

    Step 1b — Comparative pass: sample how the codebase already handles this concern
    (neighbouring files, shared helpers, established conventions). Flag *deviations*
    from the existing pattern — not the mere absence of a pattern you would prefer.
    A change that matches the surrounding code is not a finding.

    Step 2 — Scope (files to analyze):
    <one bulleted file path per line>

    Step 3 — Return findings as a JSON array with this exact shape, nothing else:
    [
      {
        "file": "src/x.ts",
        "line": 42,
        "category": "smell|error-handling|test-gap|...",
        "severity": "critical|major|minor",
        "confidence": "high|medium|low",
        "message": "one-line summary",
        "suggested_fix": "one-line fix or null"
      },
      ...
    ]

    Do not include prose around the JSON. If you find nothing, return [].
  """
)

Security auditor exception. security-reviewer is an agent, not a skill — there is no .claude/skills/security-reviewer/SKILL.md. Dispatch it like the others, but in Step 1 point it at .claude/agents/security-reviewer.md, and have it additionally apply the false-positive filter in .claude/skills/_shared/blocks/security-fp-precedents.md before emitting findings. It still returns the same JSON array (use category values like injection/auth/exposure/config); the agent's own markdown output format does not apply inside the pipeline.

Optional adversarial lens. Pass adversarial (or adversarial:true) in the arguments to add the devils-advocate agent (.claude/agents/devils-advocate.md) as an extra lens that tries to falsify the change rather than confirm it (unstated assumptions, breaking inputs, quiet reinterpretations). Off by default — every other lens checks for goodness; this one pushes back. It returns a ranked objection list; fold its would-ship-broken/risky items into the report as correctness/major findings. Distinct from verify (which knocks down existing findings); adversarial generates new ones.

Wait for all Tasks to complete before continuing. If any single Task errors out, record the failure under "Skipped audits" and continue with the rest — never abort the whole pipeline because one auditor failed.

Phase 4: Dedupe + Confidence Gating

Merge findings using this algorithm:

1. Parse each auditor's JSON into a flat array, tagging each finding with audit: <auditor-name>.

2. Group findings by the tuple (normalized_file, line_bucket, category) where:

   • normalized_file = repo-relative path with leading ./ stripped
   • line_bucket = floor(line / 5) * 5 — collapses findings within 5 lines of each other
   • category = the auditor's category string, lowercased

3. Within each group:

   • Merge messages: keep the longest, most specific one as primary; append distinct shorter ones as bullets under "Also noted"
   • Compute provenance = sorted unique list of audit tags in the group
   • Bump confidence: if |provenance| ≥ 2, set group confidence to high regardless of individual confidences
   • Max severity wins: group severity = highest of (critical > major > minor)

4. Confidence gate (drop low-signal noise):

   • Keep group if |provenance| ≥ 2 — multiple auditors agreed
   • Else keep group if confidence = high and severity ∈ {critical, major}
   • Else keep group if confidence = high and the message names a concrete file:line (not a category-only observation)
   • Else drop — single low/medium-confidence finding is noise

5. Sort by (severity desc, |provenance| desc, file asc).

Phase 4.5: Adversarial Verification (opt-in)

By default the pipeline trusts each auditor's self-reported confidence (Phase 4 gates structurally). Pass verify (or verify:true) in the arguments to add an independent second pass that tries to knock down each surviving finding before it reaches the report. This is the highest-precision mode and the most token-expensive — it spends one extra subagent per verified finding, so it is off by default.

When verify is set, after Phase 4:

1. Select what to verify. Take surviving findings of severity critical or major (skip minor — low stakes, not worth the spend). Cap at the top 15 by severity; if more survive, verify those 15 and carry the rest through with their Phase 4 status, marked unverified in the report.

2. Verify each one independently. Dispatch one Task per finding in a single message (parallel), capped at 8 concurrent — drain and refill until done. Each verifier gets only the finding's {file, line, category, severity, one-line claim} plus the code it reads itself — never the other findings, and never the originating auditor's reasoning. That isolation is the point: a verifier that can't see the auditor's argument (or its peers' verdicts) can't be talked into agreeing, so it has to reproduce the judgement from the code or drop it. (The auditors only ever emit a one-line message, so there is no chain-of-thought to leak — keep it that way; don't pass the auditor's working into the verifier prompt.)

   Task(
     description: "verify <category> finding at <file>:<line>",
     subagent_type: "general-purpose",
     prompt: """
       Independently assess ONE candidate finding. Reproduce the judgement from the
       code itself — do not trust the claim.
   
       Finding: <file>:<line> · <category> · <severity>
       Claim: <message>
   
       Standard to judge against:
       - For security categories (injection|auth|exposure|config), read and apply
         `.claude/skills/_shared/blocks/security-fp-precedents.md`.
       - Otherwise, read `.claude/skills/<originating-audit>/SKILL.md` and judge
         against its criteria.
   
       Read the cited file and enough surrounding code to trace the real path. Is
       this a concrete, exploitable/actionable issue — not theoretical, not a style
       preference, not excluded by the standard above?
   
       Return this JSON and nothing else:
       { "confidence": 1-10, "verdict": "keep" | "drop", "reason": "one line" }
     """
   )

3. Gate on the verifier. Drop any finding the verifier scored below 8. Attach the surviving verifier confidence + reason to each kept finding.

   Critical findings get a 3-vote majority. A single verifier is enough signal for major findings — bounded stakes, bounded spend — but a critical gates the merge, so verify each critical with 3 independent verifiers (same isolated-context rules; dispatch the three in the same parallel batch) and keep it only if ≥2 of 3 score it ≥8. A 1-of-3 or 0-of-3 result drops the finding; a 2-of-3 keeps it but records the split in the report. This spends 3× on the handful of findings that most warrant it and stays single-pass everywhere else — the majority vote is what a lone verifier's own miscalls can't give you.

4. Verification is read-only and never modifies code. If a verification Task errors, keep the finding with its Phase 4 confidence and mark it unverified — never drop a finding because its verifier crashed.

Phase 5: Assemble the Report

Produce one markdown report and offer to save it. Do not fix issues; only report.

Optional save target: tasks/reviews/<YYYY-MM-DD>-<scope-slug>.md. Ask the user before writing if any review file already exists for the same scope today; otherwise just create it and tell them the path.

Output Format

# Review Pipeline Report

**Scope:** <N files | git range | path glob>
**Audits run:** <comma-separated names> · <X> in parallel
**Audits skipped:** <comma-separated, or "none">
**Findings:** <C critical, M major, N minor> after dedupe (<R> raw)
**Verification:** <V verified · D dropped below threshold · U unverified> — include only when `verify` ran


## Cross-Audit Findings (≥2 auditors agreed)

> Highest signal — these are flagged by multiple lenses, treat as high confidence.

| # | Severity | File:Line | Category | Auditors | Issue |
|---|----------|-----------|----------|----------|-------|
| 1 | critical | src/auth.ts:42 | error-handling | quality, testing | ... |

## Single-Audit Findings — Critical

| # | File:Line | Audit | Issue | Suggested Fix |
|---|-----------|-------|-------|---------------|

## Single-Audit Findings — Major

| # | File:Line | Audit | Issue | Suggested Fix |
|---|-----------|-------|-------|---------------|

## Single-Audit Findings — Minor

> Folded by default. Expand only if reviewing thoroughly.

<details>
<summary>N minor findings</summary>

| # | File:Line | Audit | Issue |
|---|-----------|-------|-------|

</details>

## Skipped Audits

| Audit | Reason |
|-------|--------|
| dependency-audit | no dep file in scope |
| performance-audit | parallel cap reached; lower fit than selected 5 |

## Suggested Next Action

1. Address all Cross-Audit Findings first — they have the strongest signal
2. Triage Critical single-audit findings
3. Consider whether any Major findings warrant a follow-up task in `tasks/todo.md`
4. If patterns recur across PRs, promote them to `tasks/lessons/` or a hook

Report Guidelines

• File paths with line numbers (file.ts:42) for every finding — never bare filenames
• "Also noted" bullets under a primary finding when auditors phrased the same issue differently
• The Cross-Audit table is the headline — readers scan it first
• Skipped audits get one row each — never silently omit
• Minor findings collapsed into a <details> block to keep the report scannable
• When verify ran, append the verifier score to each verified finding (e.g., ✓ 9/10) and list what was dropped below threshold under a short "Dropped in verification" note — surfacing what was filtered is part of the signal

Run Mode

This skill supports interactive (default) and headless modes — see the canonical contract in .claude/skills/_shared/blocks/mode-detection.md.

Headless detection: presence of mode:headless in arguments. Other tokens after the flag are treated as explicit audit list (e.g., mode:headless audits:testing,security).

verify / verify:true is a reserved flag token (enables Phase 4.5) — not an audit name. Strip it before parsing the audit list, the same way mode: tokens are stripped.

Headless review-pipeline is suitable for: scheduled PR sweeps (/loop /review-pipeline mode:headless), CI-side pre-merge gates, and skill-to-skill orchestration where the parent skill chose the scope.

Notes

• This skill orchestrates; it does not analyze. Quality of the report depends on the underlying audit skills.
• Default cap is 5 parallel audits to bound token spend. Override with explicit selection if you need more.
• Confidence bump is asymmetric: agreement promotes to high, but disagreement does not demote. Two auditors disagreeing on severity → take the higher severity.
• The line bucket of 5 is a heuristic for "same issue, different line counted". If you need stricter matching (exact line), say so when invoking; if you need looser (same function), increase the bucket.
• Don't run on the whole repo by default. If scope resolution returns >100 files, ask the user to narrow.
• Save the report only when useful. Quick checks during development don't need a saved file — interactive feedback is the point.
• Adversarial verification (verify) is opt-in and costs one extra subagent per verified finding. It's bounded (critical/major only, top 15, 8 concurrent) but still the most expensive mode — reach for it on high-stakes diffs (auth, releases) or when someone disputes the findings, not on every run.
• Comparative pass (Step 1b) is always on and free — it sharpens every auditor by anchoring findings to deviations from the codebase's own patterns rather than abstract ideals.
• The verifier's "not-a-bug" standard is codified, not improvised. Each verifier judges against a fixed rubric — the originating audit's own SKILL.md criteria, and for security categories the .claude/skills/_shared/blocks/security-fp-precedents.md precedent list — so false-positive exclusion is uniform across runs instead of per-verifier taste. Extend those files, not the verifier prompt, when a class of false positive keeps surviving.
• This is a reviewer → dedupe → verify chain, deliberately not a fix loop. The pipeline chains lenses adversarially (auditors → provenance-gated merge → independent verifiers) but stays report-only — it never proposes or applies fixes, so nothing it emits can be a fix it also graded. Turning a finding into a change (and re-reviewing that change) is a separate step: hand the report to /ship's verification gate or a /deepening-review fix pass, where the edit gets its own review rather than being adjudicated by the same run that proposed it.