← Explore all skills

AGENT DEVELOPMENT / OpenAI

OpenAI Evaluate Skill

Connects a local skill review to structural checks, budget explanations and an explicitly separate live benchmark workflow.

“Evaluate a local Codex skill in engineer-friendly terms.…”

01 / THE REASONING

Why this made the selection.

  • Distinguishes static instruction-size estimates from observed usage collected by real agent runs.
  • Directs attention to broken references, metadata and helper code before proposing a benchmark.

02 / THE REVIEW RECORD

What we actually inspected.

Source review has boundaries.
A clear record is more useful than a “safe” badge.

Material inspected

  • plugins/plugin-eval/skills/evaluate-skill/SKILL.md
  • plugins/plugin-eval/references/chat-first-workflows.md
  • plugins/plugin-eval/src/core/analyze.js
  • plugins/plugin-eval/src/cli.js
  • plugins/plugin-eval/.codex-plugin/plugin.json (license MIT, no licence file beside it)
  • plugins/plugin-eval/README.md

Our findings

  • The inspected analyzer combines structural evaluation, code checks, budget profiles and optional observed-usage input.
  • The workflow distinguishes analyze from init-benchmark and requires reviewing the generated scenarios before live runs.
  • The publisher documents local reports and benchmark artifacts under .plugin-eval; this catalogue did not run them.
  • The SKILL.md drives the plugin-eval CLI: start, analyze, explain-budget, measurement-plan, init-benchmark and benchmark --dry-run. The README requires Node.js >=20 and says the package is private, so it runs from a checkout.
  • Re-pinned from 1dc195897af4161d039b80d8471ec0a10c9bbc89 to 0722921d5542fc593105c27bd52630babd8b8c2a: no file under plugins/plugin-eval changed between the two (GitHub compare). plugin.json still declares license MIT, and no licence file exists in the plugin or at the repository root at the new pin.

Not established by this review

  • Source inspection only; no skill installation or workflow execution was performed.
  • Not executed: it was not installed or run.

The review applies to the material and revision named here. A newer upstream release can change its behavior.

03 / PUT IT TO WORK

Add a skill to your workflow.

Upstream setup instructions ↗
  1. Use a checkout of the complete plugins/plugin-eval directory and follow its README setup.
  2. From that directory, node ./scripts/plugin-eval.js --help runs the CLI without a global install; npm link is the documented optional global command.
  3. Install the full Codex plugin through the documented local marketplace instructions for natural-language skill invocation.

Before you start

  • Node.js meeting the documented runtime requirement
  • Complete plugin files; a live Codex CLI session is needed only for benchmarks
  • Complete plugin checkout and its CLI dependencies
  • Codex CLI access only for live benchmarking

04 / FOLLOW THE EVIDENCE

The source trail.

Our notes are separate from the original resource.
Check upstream before adopting a new version.

  1. https://github.com/openai/plugins/blob/0722921d5542fc593105c27bd52630babd8b8c2a/plugins/plugin-eval/skills/evaluate-skill/SKILL.md

    Supports: summary, upstreamDescription, whySelected, bestFor, limitations, review, compatibility, access

  2. https://github.com/openai/plugins/blob/0722921d5542fc593105c27bd52630babd8b8c2a/plugins/plugin-eval/references/chat-first-workflows.md

    Supports: review, limitations, access

  3. https://github.com/openai/plugins/blob/0722921d5542fc593105c27bd52630babd8b8c2a/plugins/plugin-eval/src/core/analyze.js

    Supports: review, limitations, access

  4. https://github.com/openai/plugins/blob/0722921d5542fc593105c27bd52630babd8b8c2a/plugins/plugin-eval/src/cli.js

    Supports: review, limitations, access

  5. https://github.com/openai/plugins/blob/0722921d5542fc593105c27bd52630babd8b8c2a/plugins/plugin-eval/.codex-plugin/plugin.json

    Supports: license

  6. https://github.com/openai/plugins/blob/0722921d5542fc593105c27bd52630babd8b8c2a/README.md

    Supports: install, compatibility, access

  7. https://github.com/openai/plugins/blob/0722921d5542fc593105c27bd52630babd8b8c2a/plugins/plugin-eval/README.md

    Supports: install, compatibility, access, review

  8. https://raw.githubusercontent.com/openai/plugins/0722921d5542fc593105c27bd52630babd8b8c2a/plugins/plugin-eval/skills/evaluate-skill/SKILL.md

    Supports: bestFor, review.notTested, upstreamDescription, review.reviewedAt

Evidence & Ask