TESTING AND RELIABILITY / Trail of Bits
Vector Forge
Finds implementations of a cryptographic algorithm, runs mutation testing to locate escaped mutants, generates test vectors that exercise the uncovered paths, and compares mutation kill rates before and after.
“Mutation-driven test vector generation. Finds implementations of a cryptographic algorithm or protocol, runs mutation testing to identify escaped mutants, then generates new test vectors that deliberately exercise the uncovered code paths. Compares before/after mutation kill rates to prove vector effectiveness. Use when generating cryptographic test vectors, measuring Wycheproof coverage gaps, finding escaped mutants via mutation testing, creating cross-implementation test suites, or improving test vector coverage for crypto primitives.”
01 / THE REASONING
Why this made the selection.
- Finds a crypto implementation, runs mutation testing to identify escaped mutants, then generates test vectors that exercise the uncovered code paths.
- Measures vector effectiveness by re-running mutation testing and comparing kill rates, and writes VECTOR_FORGE_REPORT.md with the before/after delta.
02 / THE REVIEW RECORD
What we actually inspected.
Source review has boundaries.
A clear record is more useful than a “safe” badge.
Material inspected
- plugins/trailmark/skills/vector-forge/SKILL.md
- plugins/trailmark/skills/vector-forge/references/mutation-frameworks.md
- plugins/trailmark/README.md (prerequisites and version notes)
- plugins/trailmark/agents/code-slice-worker.md (host note)
- README.md (installation and licence sections)
- LICENSE
Our findings
- Vector Forge's SKILL.md frontmatter declares no allowed-tools.
- The workflow runs discovery, harness, baseline, escape analysis, vector generation and validation, ending in a coverage report and new vectors.
- VECTOR_FORGE_REPORT.md covers the baseline, escape analysis, new vectors, after results and the before/after delta.
Not established by this review
- Not executed: it was not installed or run.
- The Vector Forge skill and any bundled scripts were not run.
The review applies to the material and revision named here. A newer upstream release can change its behavior.
03 / PUT IT TO WORK
Add a skill to your workflow.
- `/plugin marketplace add trailofbits/skills` (repository README) adds the whole Trail of Bits marketplace, not one skill.
- Install the `trailmark` plugin from `/plugin menu`; it bundles the Vector Forge skill with the other trailmark skills listed in the trailmark plugin README.
- `uv tool install trailmark` (trailmark plugin README) installs the Trailmark CLI that this skill drives.
Before you start
- The Trailmark CLI, installed with `uv tool install trailmark` (trailmark plugin README)
- uv
- At least one implementation of the target algorithm in a language with mutation-testing support
- A test harness that consumes test vectors
- A mutation-testing framework for the target language; the skill installs one if it is missing
04 / FOLLOW THE EVIDENCE
The source trail.
Our notes are separate from the original resource.
Check upstream before adopting a new version.
- Vector Forge SKILL.md ↗Checked
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/trailmark/skills/vector-forge/SKILL.md
Supports: upstreamDescription, summary, whySelected, bestFor, limitations, access, review.findings
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/trailmark/skills/vector-forge/references/mutation-frameworks.md
Supports: limitations, access.permissions
- trailmark plugin README ↗Checked
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/trailmark/README.md
Supports: install.prerequisites, install.steps
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/trailmark/agents/code-slice-worker.md
Supports: compatibility
- trailofbits/skills README ↗Checked
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/README.md
Supports: install.steps, license
- trailofbits/skills LICENSE ↗Checked
https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/LICENSE
Supports: license