---
title: "Testing Handbook Skill Generator: review, setup & limitations · Undominated.ai"
canonical: https://undominated.ai/skills/trailofbits-testing-handbook-generator/
description: "Maintainer workflow that reads sections of the Trail of Bits Testing Handbook (appsec.guide) and writes or refreshes the matching Claude Code skills inside the trailofbits/skills testing-handbook-skills plugin."
---

# Testing Handbook Skill Generator: review, setup & limitations · Undominated.ai

> Maintainer workflow that reads sections of the Trail of Bits Testing Handbook (appsec.guide) and writes or refreshes the matching Claude Code skills inside the trailofbits/skills testing-handbook-skills plugin.

[← Explore all skills](/skills/)

AGENT DEVELOPMENT / Trail of Bits

# Testing Handbook Skill Generator

Maintainer workflow that reads sections of the Trail of Bits Testing Handbook (appsec.guide) and writes or refreshes the matching Claude Code skills inside the trailofbits/skills testing-handbook-skills plugin.

 [See setup guidance ↓](#setup)[Original source ↗](https://github.com/trailofbits/skills/tree/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator)

SOURCE REVIEW

 Reviewed 2026-10-22
 Evidence 8 linked sources
 Publisher Trail of Bits
 Licence [CC-BY-SA-4.0 ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/LICENSE)
 Revision 8e54050ae63c
 [Read what was—and wasn’t—checked ↓](#review)

“Generates Claude Code skills from the Trail of Bits Testing Handbook (appsec.guide), analyzing handbook pages and emitting SKILL.md files with the structure each skill type requires. Use when creating or refreshing a skill from handbook content, or when the user names the testing handbook or appsec.guide. Not for answering security testing questions — the generated skills cover those.”

 [Trail of Bits · upstream description ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/SKILL.md) Our analysis follows below.

01 / THE REASONING

## Why this made the selection.

 - It maps handbook sections to four skill types (tool, fuzzer, technique and domain), each with its own template under templates/.
- Generation runs in two passes, content first and Related Skills links second, and the plugin's validate-skills.py then checks frontmatter, required sections, line limits and that cross-referenced skills exist.
- It presents a plan before generating anything and confines writes to the testing-handbook-skills plugin and the repository README.

### A good fit for

 - Regenerate one handbook-derived skill, such as libfuzzer from fuzzing/c-cpp/10-libfuzzer, after the handbook section changes
- Plan which Testing Handbook sections should become new skills, skipping drafts, empty directories and GUI-only tools
- Re-check generated SKILL.md files for structure and broken cross-references with the plugin's validator

### Weigh up before choosing

 - It maintains the trailofbits/skills repository itself: it writes under plugins/testing-handbook-skills/skills/ and edits the repository root README.md, so it needs a checkout of that repository.
- Validation calls plugins/testing-handbook-skills/scripts/validate-skills.py, which sits outside the skill directory; a copy of the skill directory alone does not include it.
- It needs a local trailofbits/testing-handbook clone. If none is in ./, ../ or ~/testing-handbook, it asks for a path and, if the user agrees, runs git clone --depth=1.
- Per generated skill it fetches up to 5 external URLs linked from the handbook with WebFetch, 30 seconds each; video links are kept as titles only.
- It is not for answering security-testing questions; its description sends those to the generated skills.
- GUI-only handbook tools, such as web/burp, are skipped.
- The plugin README's `claude skills install testing-handbook-skills` would bring in the whole plugin, not this generator alone.
- Last commit touching plugins/testing-handbook-skills/skills/testing-handbook-generator at the pinned revision: 2026-09-16.
- Not executed: it was not installed or run.

02 / THE REVIEW RECORD

## What we actually inspected.

Source review has boundaries. A clear record is more useful than a “safe” badge.

### Material inspected

 - plugins/testing-handbook-skills/skills/testing-handbook-generator/SKILL.md
- plugins/testing-handbook-skills/skills/testing-handbook-generator/discovery.md
- plugins/testing-handbook-skills/skills/testing-handbook-generator/agent-prompt.md (process and limits)
- plugins/testing-handbook-skills/skills/testing-handbook-generator/testing.md (prerequisites and validator commands)
- plugins/testing-handbook-skills/scripts/validate-skills.py (header)
- plugins/testing-handbook-skills/README.md
- plugins/testing-handbook-skills/.claude-plugin/plugin.json
- README.md (installation, plugin table and licence sections)
- LICENSE

### Our findings

 - The frontmatter declares no allowed-tools.
- Scope Restrictions say never to modify or analyze other plugins, and to generate only from handbook content and resources it references.
- Pass 1 runs one agent per skill in parallel and leaves a Related Skills placeholder; Pass 2 runs sequentially and replaces it with links.
- A single failed agent is re-run on its own; the skill says not to re-run the whole batch.
- Generated SKILL.md files have a hard limit of 500 lines, with splitting from 450.
- validate-skills.py declares pyyaml>=6.0 as PEP 723 inline metadata, which uv resolves on first use.
- Self-improvement is part of the workflow: after a run it may edit its own SKILL.md, templates, discovery.md and testing.md.

### Not established by this review

 - Not executed: it was not installed or run.
- No handbook was cloned, no skill was generated and the validator was not run.

The review applies to the material and revision named here. A newer upstream release can change its behavior.

03 / PUT IT TO WORK

## Add a skill to your workflow.

[Upstream setup instructions ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/README.md)
 - Add the marketplace with `/plugin marketplace add trailofbits/skills`. This lists the full Trail of Bits collection; it does not install the generator.
- From `/plugin menu`, install testing-handbook-skills. That plugin carries this generator together with the handbook-derived skills already generated into it, so it is not the generator alone.
- Run it from a checkout of trailofbits/skills, because every path it writes is inside that repository.

### Before you start

 - A checkout of trailofbits/skills
- uv, for `uv run scripts/validate-skills.py`
- Git, and a local trailofbits/testing-handbook clone or the user's consent to clone one

### Compatibility

Claude Code

### Reads the handbook and linked pages; writes skills and README in the trailofbits/skills checkout

 - The SKILL.md frontmatter lists no allowed-tools, so the host's tool settings apply
- Read a local Testing Handbook checkout, or clone github.com/trailofbits/testing-handbook after the user agrees
- Fetch non-video external resources linked from handbook pages (WebFetch)
- Launch parallel generation agents, one per skill
- Write SKILL.md files under plugins/testing-handbook-skills/skills/, edit the repository root README.md, and edit its own files
- Run the plugin's validator through uv, which downloads pyyaml on first use

### Cost model

CC-BY-SA-4.0. The source names no paid service; generation launches one agent per skill in the host.

04 / FOLLOW THE EVIDENCE

## The source trail.

Our notes are separate from the original resource. Check upstream before adopting a new version.

 - [Testing Handbook Skill Generator SKILL.md ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/SKILL.md) Checked 2026-10-22 https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/SKILL.md Supports: upstreamDescription, summary, whySelected, bestFor, limitations, access, review.findings
- [Handbook discovery methodology ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/discovery.md) Checked 2026-10-22 https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/discovery.md Supports: limitations, access, install.prerequisites
- [Generation agent prompt ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/agent-prompt.md) Checked 2026-10-22 https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/agent-prompt.md Supports: limitations, access
- [Generated-skill testing strategy ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/testing.md) Checked 2026-10-22 https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/skills/testing-handbook-generator/testing.md Supports: install.prerequisites, whySelected
- [Plugin validator validate-skills.py ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/scripts/validate-skills.py) Checked 2026-10-22 https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/scripts/validate-skills.py Supports: whySelected, limitations, review.findings, access
- [testing-handbook-skills plugin README ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/README.md) Checked 2026-10-22 https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/testing-handbook-skills/README.md Supports: compatibility, limitations, install.steps
- [trailofbits/skills README ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/README.md) Checked 2026-10-22 https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/README.md Supports: install, license
- [trailofbits/skills LICENSE ↗](https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/LICENSE) Checked 2026-10-22 https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/LICENSE Supports: license

KEEP COMPARING

## Other approaches to consider.

Related by category or shared topics. These are alternatives to inspect, not a measured quality order.

 [### Anthropic Skill Creator ↗ Develops skills from realistic task fixtures and baseline comparisons, with separate trigger-description experiments and human output review.](/skills/anthropics-skill-creator/)[### Docker Agent Configuration ↗ Build Docker Agent configurations with per-agent toolsets, provider credentials, delegated teams and bounded autonomy.](/skills/docker-agent-config/)[### OpenAI Evaluate Skill ↗ Connects a local skill review to structural checks, budget explanations and an explicitly separate live benchmark workflow.](/skills/openai-evaluate-skill/)

 [AI Tools ↗](/tools/)[Skills ↗](/skills/)[Agents ↗](/agents/)[MCP Servers ↗](/mcp-servers/)[Workflows ↗](/workflows/)

## Continue your investigation

 - [Choose a toolkit](/tools/)
- [Inspect execution hosts](/agents/)
- [Check tool access](/mcp-servers/)
- [Check project dependencies](/check-repo/)
