TESTING AND RELIABILITY / Matt Pocock
Diagnosing Bugs
A debugging procedure that establishes a reproducible symptom, tests competing explanations and preserves a regression check.
“Diagnosis loop for hard bugs and performance regressions. Use when the user says "diagnose"/"debug this", or reports something broken/throwing/failing/slow.”
01 / THE REASONING
Why this made the selection.
- Requires a red-capable reproduction before edits, so an apparent improvement has a concrete behavior to compare against.
- Tests one hypothesis at a time and asks for the original scenario to be rerun after the fix, including removal of temporary instrumentation.
02 / THE REVIEW RECORD
What we actually inspected.
Source review has boundaries.
A clear record is more useful than a “safe” badge.
Material inspected
- skills/engineering/diagnosing-bugs/SKILL.md
- README.md (installation and setup sections; the README has no licence section)
- LICENSE (MIT, copyright 2026 Matt Pocock)
- skills/engineering/diagnosing-bugs/agents/openai.yaml
- skills/engineering/diagnosing-bugs/scripts/hitl-loop.template.sh
Our findings
- The definition distinguishes observations from hypotheses and ties each proposed mechanism to a falsification test.
- The full HITL helper was inspected: it prints user-supplied answers and needs adaptation to the real scenario.
- A Redact section comes first: secrets in shown commands, outputs and captured artifacts are replaced with <REDACTED>, and loops read credentials from environment variables.
- Phase 1 ends only when one command already run is red-capable, deterministic, fast and agent-runnable. Phase 3 needs 3–5 ranked, falsifiable hypotheses shown to the user before testing, and every debug log carries a unique prefix such as [DEBUG-a4f2] so cleanup is one grep.
- Re-pinned from c55ee46073ed923f86ce59a5eb3b6d895095d1b7 to 49dd158d1076134a641b33efb035946536778336: SKILL.md now reads GLOSSARY.md instead of CONTEXT.md, and Phase 5 adds a diff against a pristine copy when the red was forced by mutating code or a fixture. scripts/hitl-loop.template.sh and agents/openai.yaml are unchanged.
Not established by this review
- No upstream definition, helper, installer or generated workload was executed.
- No end-to-end agent session or effectiveness benchmark was run.
- Not executed: it was not installed or run.
The review applies to the material and revision named here. A newer upstream release can change its behavior.
03 / PUT IT TO WORK
Add a skill to your workflow.
DOCUMENTED COMMAND
npx skills add mattpocock/skills/skills/engineering/diagnosing-bugs --skill diagnosing-bugs Run in your terminal to choose your AI tools, project or global scope, and installation method. This short command follows the current upstream version; the review covers the revision shown above.
Optional: install the reviewed revision
Keeps the original source and installer version where recorded, while leaving your AI tools and installation scope for you to choose.
npx skills add https://github.com/mattpocock/skills/tree/49dd158d1076134a641b33efb035946536778336/skills/engineering/diagnosing-bugs --skill diagnosing-bugs - Install the complete skill directory, including its references and helpers; select your AI tools and project or global scope.
- Review the instructions before invoking them. The source revision shown here is the one reviewed; the installer was not executed or version-pinned by this review.
Before you start
- Node.js/npm and git for the documented Skills installer
- An agent host that supports the skill format and the tools required by the workflow
04 / FOLLOW THE EVIDENCE
The source trail.
Our notes are separate from the original resource.
Check upstream before adopting a new version.
https://github.com/mattpocock/skills/blob/49dd158d1076134a641b33efb035946536778336/skills/engineering/diagnosing-bugs/SKILL.md
Supports: summary, upstreamDescription, whySelected, bestFor, limitations, review, access
https://github.com/mattpocock/skills/blob/49dd158d1076134a641b33efb035946536778336/README.md
Supports: install, compatibility
https://github.com/mattpocock/skills/blob/49dd158d1076134a641b33efb035946536778336/LICENSE
Supports: license, access.cost
https://github.com/mattpocock/skills/blob/49dd158d1076134a641b33efb035946536778336/skills/engineering/diagnosing-bugs/agents/openai.yaml
Supports: review
https://github.com/mattpocock/skills/blob/49dd158d1076134a641b33efb035946536778336/skills/engineering/diagnosing-bugs/scripts/hitl-loop.template.sh
Supports: review, limitations
https://github.com/vercel-labs/skills/blob/7407f3893ad4dceab546ac002c3ef806e4000c73/README.md
Supports: install
https://github.com/vercel-labs/skills/blob/7407f3893ad4dceab546ac002c3ef806e4000c73/src/git.ts
Supports: install
https://github.com/vercel-labs/skills/blob/7407f3893ad4dceab546ac002c3ef806e4000c73/src/source-parser.ts
Supports: install
https://raw.githubusercontent.com/mattpocock/skills/49dd158d1076134a641b33efb035946536778336/skills/engineering/diagnosing-bugs/SKILL.md
Supports: bestFor, review
https://raw.githubusercontent.com/mattpocock/skills/49dd158d1076134a641b33efb035946536778336/README.md
Supports: limitations