AGENT DEVELOPMENT / Trail of Bits
Testing Handbook Skill Generator
Maintainer workflow that reads sections of the Trail of Bits Testing Handbook (appsec.guide) and writes or refreshes the matching Claude Code skills inside the trailofbits/skills testing-handbook-skills plugin.
“Generates Claude Code skills from the Trail of Bits Testing Handbook (appsec.guide), analyzing handbook pages and emitting SKILL.md files with the structure each skill type requires. Use when creating or refreshing a skill from handbook content, or when the user names the testing handbook or appsec.guide. Not for answering security testing questions — the generated skills cover those.”
01 / THE REASONING
Why this made the selection.
- It maps handbook sections to four skill types (tool, fuzzer, technique and domain), each with its own template under templates/.
- Generation runs in two passes, content first and Related Skills links second, and the plugin's validate-skills.py then checks frontmatter, required sections, line limits and that cross-referenced skills exist.
- It presents a plan before generating anything and confines writes to the testing-handbook-skills plugin and the repository README.
02 / THE REVIEW RECORD
What we actually inspected.
Source review has boundaries.
A clear record is more useful than a “safe” badge.
Material inspected
- plugins/testing-handbook-skills/skills/testing-handbook-generator/SKILL.md
- plugins/testing-handbook-skills/skills/testing-handbook-generator/discovery.md
- plugins/testing-handbook-skills/skills/testing-handbook-generator/agent-prompt.md (process and limits)
- plugins/testing-handbook-skills/skills/testing-handbook-generator/testing.md (prerequisites and validator commands)
- plugins/testing-handbook-skills/scripts/validate-skills.py (header)
- plugins/testing-handbook-skills/README.md
- plugins/testing-handbook-skills/.claude-plugin/plugin.json
- README.md (installation, plugin table and licence sections)
- LICENSE
Our findings
- The frontmatter declares no allowed-tools.
- Scope Restrictions say never to modify or analyze other plugins, and to generate only from handbook content and resources it references.
- Pass 1 runs one agent per skill in parallel and leaves a Related Skills placeholder; Pass 2 runs sequentially and replaces it with links.
- A single failed agent is re-run on its own; the skill says not to re-run the whole batch.
- Generated SKILL.md files have a hard limit of 500 lines, with splitting from 450.
- validate-skills.py declares pyyaml>=6.0 as PEP 723 inline metadata, which uv resolves on first use.
- Self-improvement is part of the workflow: after a run it may edit its own SKILL.md, templates, discovery.md and testing.md.
Not established by this review
- Not executed: it was not installed or run.
- No handbook was cloned, no skill was generated and the validator was not run.
The review applies to the material and revision named here. A newer upstream release can change its behavior.
03 / PUT IT TO WORK
Add a skill to your workflow.
- Add the marketplace with `/plugin marketplace add trailofbits/skills`. This lists the full Trail of Bits collection; it does not install the generator.
- From `/plugin menu`, install testing-handbook-skills. That plugin carries this generator together with the handbook-derived skills already generated into it, so it is not the generator alone.
- Run it from a checkout of trailofbits/skills, because every path it writes is inside that repository.
Before you start
- A checkout of trailofbits/skills
- uv, for `uv run scripts/validate-skills.py`
- Git, and a local trailofbits/testing-handbook clone or the user's consent to clone one
04 / FOLLOW THE EVIDENCE
The source trail.
Our notes are separate from the original resource.
Check upstream before adopting a new version.
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/SKILL.md
Supports: upstreamDescription, summary, whySelected, bestFor, limitations, access, review.findings
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/discovery.md
Supports: limitations, access, install.prerequisites
- Generation agent prompt ↗Checked
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/agent-prompt.md
Supports: limitations, access
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/testing.md
Supports: install.prerequisites, whySelected
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/scripts/validate-skills.py
Supports: whySelected, limitations, review.findings, access
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/README.md
Supports: compatibility, limitations, install.steps
- trailofbits/skills README ↗Checked
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/README.md
Supports: install, license
- trailofbits/skills LICENSE ↗Checked
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/LICENSE
Supports: license