← Explore all skills

TESTING AND RELIABILITY / Trail of Bits

Vector Forge

Finds implementations of a cryptographic algorithm, runs mutation testing to locate escaped mutants, generates test vectors that exercise the uncovered paths, and compares mutation kill rates before and after.

“Mutation-driven test vector generation. Finds implementations of a cryptographic algorithm or protocol, runs mutation testing to identify escaped mutants, then generates new test vectors that deliberately exercise the uncovered code paths. Compares before/after mutation kill rates to prove vector effectiveness. Use when generating cryptographic test vectors, measuring Wycheproof coverage gaps, finding escaped mutants via mutation testing, creating cross-implementation test suites, or improving test vector coverage for crypto primitives.”

01 / THE REASONING

Why this made the selection.

  • Finds a crypto implementation, runs mutation testing to identify escaped mutants, then generates test vectors that exercise the uncovered code paths.
  • Measures vector effectiveness by re-running mutation testing and comparing kill rates, and writes VECTOR_FORGE_REPORT.md with the before/after delta.

02 / THE REVIEW RECORD

What we actually inspected.

Source review has boundaries.
A clear record is more useful than a “safe” badge.

Material inspected

  • plugins/trailmark/skills/vector-forge/SKILL.md
  • plugins/trailmark/skills/vector-forge/references/mutation-frameworks.md
  • plugins/trailmark/README.md (prerequisites and version notes)
  • plugins/trailmark/agents/code-slice-worker.md (host note)
  • README.md (installation and licence sections)
  • LICENSE

Our findings

  • Vector Forge's SKILL.md frontmatter declares no allowed-tools.
  • The workflow runs discovery, harness, baseline, escape analysis, vector generation and validation, ending in a coverage report and new vectors.
  • VECTOR_FORGE_REPORT.md covers the baseline, escape analysis, new vectors, after results and the before/after delta.

Not established by this review

  • Not executed: it was not installed or run.
  • The Vector Forge skill and any bundled scripts were not run.

The review applies to the material and revision named here. A newer upstream release can change its behavior.

03 / PUT IT TO WORK

Add a skill to your workflow.

Upstream setup instructions ↗
  1. `/plugin marketplace add trailofbits/skills` (repository README) adds the whole Trail of Bits marketplace, not one skill.
  2. Install the `trailmark` plugin from `/plugin menu`; it bundles the Vector Forge skill with the other trailmark skills listed in the trailmark plugin README.
  3. `uv tool install trailmark` (trailmark plugin README) installs the Trailmark CLI that this skill drives.

Before you start

  • The Trailmark CLI, installed with `uv tool install trailmark` (trailmark plugin README)
  • uv
  • At least one implementation of the target algorithm in a language with mutation-testing support
  • A test harness that consumes test vectors
  • A mutation-testing framework for the target language; the skill installs one if it is missing

04 / FOLLOW THE EVIDENCE

The source trail.

Our notes are separate from the original resource.
Check upstream before adopting a new version.

  1. Vector Forge SKILL.md ↗Checked

    https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/trailmark/skills/vector-forge/SKILL.md

    Supports: upstreamDescription, summary, whySelected, bestFor, limitations, access, review.findings

  2. https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/trailmark/skills/vector-forge/references/mutation-frameworks.md

    Supports: limitations, access.permissions

  3. trailmark plugin README ↗Checked

    https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/trailmark/README.md

    Supports: install.prerequisites, install.steps

  4. https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/plugins/trailmark/agents/code-slice-worker.md

    Supports: compatibility

  5. https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/README.md

    Supports: install.steps, license

  6. https://raw.githubusercontent.com/trailofbits/skills/8e54050ae63c2f62224c35b1616efe71865b4777/LICENSE

    Supports: license

Evidence & Ask