Investigate a job that reports success but changed nothing

Hunt the silent success — a green exit code whose supposed effect never actually happened.

Also usesclaude-plugins-official

The most expensive CI defect is the green one. A job exits zero, the badge stays green, and the work it was supposed to do never happened. Conditional skips, swallowed errors, and unmatched patterns all produce this shape.

Start here

One prompt to start

This job exits zero but nothing downstream actually changed -- hunt for swallowed errors and silent skips first, then prove the wire that the job is supposed to produce an effect rather than trusting the exit code, and check whether anything would even fail if it silently did nothing again.

Needspr-review-toolkit, andon, confab

Beats

Run these in order. Each prompt is copy-pasteable straight into Claude Code.

1. pr-review-toolkit:silent-failure-hunteralso in 2 other recipes

   • Fix misleading error or diagnostic output
   • Whole-branch review without re-trusting the branch's own self-assessment

   Purpose-built for exactly this shape: silent failures, inadequate error handling, and fallback behavior that suppresses the real outcome.

   Run this beat on its own

   this job exits zero but nothing downstream changed — go hunt for swallowed errors, skipped branches, and fallbacks that hide the real outcome

2. andon:andon-verifyalso in 13 other recipes

   • Audit supply-chain pinning and dependencies
   • Finish and integrate a development branch
   • Investigate a fix that did not stick
   • Investigate tests that pass while the code is broken
   • Root-cause a bugfix from a tracked issue
   • Build a new feature in an existing codebase
   • Collapse duplication hand-synced across N sites
   • Execute a written plan across parallel workstreams
   • Migrate a return shape or type representation
   • Propagate a vendored artifact to N copies
   • Audit your own agent and skill definitions for loop-reliability defects
   • Do oracle engineering and numerical V&V
   • Whole-branch review without re-trusting the branch's own self-assessment

   A green exit code is not evidence; the wire "job ran → artifact changed" has to be proven independently.

   Run this beat on its own

   prove the wire: this job is supposed to produce a version bump. Show me evidence it did, not that it exited zero.

3. confab:confab-assertion-auditalso in 3 other recipes

   • Investigate a fix that did not stick
   • Investigate tests that pass while the code is broken
   • Do oracle engineering and numerical V&V

   If nothing asserts the job's effect, the next silent skip is invisible again.

   Run this beat on its own

   is there any test or check that would fail if this job silently did nothing?

Worked example

Grounded in — why this beat order is trustworthy

`.github/workflows/auto-version-bump.yml` skips entirely when the head commit message does not start with a recognized conventional-commit type; its own header comment records that a plain-English PR title "is silently skipped, by design". The run is green and bumps nothing — the exact shape `silent-failure-hunter` is built for.

Do / Don't

• ✓Hunt for swallowed errors, skipped branches, and fallbacks that hide the real outcome -- that's exactly this failure's shape.
• ✓Prove the job's supposed effect independently -- a green exit code by itself is not evidence anything happened.
• ✓Check whether any test or assertion would fail if the job silently did nothing -- if not, the next silent skip is invisible again.

• ×Don't trust a green exit code as proof a job did what it claims -- the most expensive CI defect is exactly the green one that changed nothing.
• ×Don't assume a conditional skip is announced -- this repo has one that's silent by its own design, documented in its own header comment.
• ×Don't leave the job's effect unasserted -- nothing catching a silent skip once means nothing will catch it the next time either.