AGENT DEVELOPMENT / OpenAI
OpenAI Evaluate Skill
Connects a local skill review to structural checks, budget explanations and an explicitly separate live benchmark workflow.
“Evaluate a local Codex skill in engineer-friendly terms. Use when the user says "evaluate this skill", "give me an analysis of the game dev skill",”
01 / THE REASONING
Why this made the selection.
- Distinguishes static instruction-size estimates from observed usage collected by real agent runs.
- Directs attention to broken references, metadata and helper code before proposing a benchmark.
02 / THE REVIEW RECORD
What we actually inspected.
Source review has boundaries.
A clear record is more useful than a “safe” badge.
Material inspected
- plugins/plugin-eval/skills/evaluate-skill/SKILL.md
- plugins/plugin-eval/references/chat-first-workflows.md
- plugins/plugin-eval/src/core/analyze.js
- plugins/plugin-eval/src/cli.js
- plugins/plugin-eval/.codex-plugin/plugin.json
- plugins/plugin-eval/README.md
Our findings
- The inspected analyzer combines structural evaluation, code checks, budget profiles and optional observed-usage input.
- The workflow distinguishes analyze from init-benchmark and requires reviewing the generated scenarios before live runs.
- The publisher documents local reports and benchmark artifacts under .plugin-eval; this catalogue did not run them.
Not established by this review
- Source inspection only; no skill installation or workflow execution was performed.
The review applies to the material and revision named here. A newer upstream release can change its behavior.
03 / PUT IT TO WORK
Add a skill to your workflow.
- Use a checkout of the complete plugins/plugin-eval directory and follow its README setup.
- From that directory, node ./scripts/plugin-eval.js --help runs the CLI without a global install; npm link is the documented optional global command.
- Install the full Codex plugin through the documented local marketplace instructions for natural-language skill invocation.
Before you start
- Node.js meeting the documented runtime requirement
- Complete plugin files; a live Codex CLI session is needed only for benchmarks
- Complete plugin checkout and its CLI dependencies
- Codex CLI access only for live benchmarking
04 / FOLLOW THE EVIDENCE
The source trail.
Our notes are separate from the original resource.
Check upstream before adopting a new version.
https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/skills/evaluate-skill/SKILL.md
Supports: summary, upstreamDescription, whySelected, bestFor, limitations, review, compatibility, access
https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/references/chat-first-workflows.md
Supports: review, limitations, access
https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/src/core/analyze.js
Supports: review, limitations, access
https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/src/cli.js
Supports: review, limitations, access
https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/.codex-plugin/plugin.json
Supports: license
https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/README.md
Supports: install, compatibility, access
https://github.com/vercel-labs/skills/blob/7407f3893ad4dceab546ac002c3ef806e4000c73/README.md
Supports: install
https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/README.md
Supports: install, compatibility, access, review