--- title: "OpenAI Evaluate Skill: review, setup & limitations · Undominated.ai" canonical: https://undominated.ai/skills/openai-evaluate-skill/ description: "Connects a local skill review to structural checks, budget explanations and an explicitly separate live benchmark workflow." --- # OpenAI Evaluate Skill: review, setup & limitations · Undominated.ai > Connects a local skill review to structural checks, budget explanations and an explicitly separate live benchmark workflow. [← Explore all skills](/skills/) AGENT DEVELOPMENT / OpenAI # OpenAI Evaluate Skill Connects a local skill review to structural checks, budget explanations and an explicitly separate live benchmark workflow. See setup guidance ↓Original source ↗ SOURCE REVIEW Reviewed 2026-09-21 Evidence 8 linked sources Publisher OpenAI Licence MIT (declared in plugin manifest) ↗ Revision 1dc195897af4 Read what was—and wasn’t—checked ↓ “Evaluate a local Codex skill in engineer-friendly terms. Use when the user says "evaluate this skill", "give me an analysis of the game dev skill",” OpenAI · upstream description ↗ Our analysis follows below. 01 / THE REASONING ## Why this made the selection. - Distinguishes static instruction-size estimates from observed usage collected by real agent runs. - Directs attention to broken references, metadata and helper code before proposing a benchmark. ### A good fit for - Reviewing a skill before adopting or revising it - Preparing realistic scenarios for measuring an installed skill ### Weigh up before choosing - Static scores and budget bands are tool heuristics, not proof of task quality or safety. - The CLI is marked private and is installed from a checkout; a skill-only copy does not supply it. - Benchmarking launches real Codex sessions and can exercise task permissions or incur usage costs. - MIT is declared in the plugin manifest, but no accompanying grant file was found; no redistribution artifact is offered. 02 / THE REVIEW RECORD ## What we actually inspected. Source review has boundaries. A clear record is more useful than a “safe” badge. ### Material inspected - plugins/plugin-eval/skills/evaluate-skill/SKILL.md - plugins/plugin-eval/references/chat-first-workflows.md - plugins/plugin-eval/src/core/analyze.js - plugins/plugin-eval/src/cli.js - plugins/plugin-eval/.codex-plugin/plugin.json - plugins/plugin-eval/README.md ### Our findings - The inspected analyzer combines structural evaluation, code checks, budget profiles and optional observed-usage input. - The workflow distinguishes analyze from init-benchmark and requires reviewing the generated scenarios before live runs. - The publisher documents local reports and benchmark artifacts under .plugin-eval; this catalogue did not run them. ### Not established by this review - Source inspection only; no skill installation or workflow execution was performed. The review applies to the material and revision named here. A newer upstream release can change its behavior. 03 / PUT IT TO WORK ## Add a skill to your workflow. Upstream setup instructions ↗ - Use a checkout of the complete plugins/plugin-eval directory and follow its README setup. - From that directory, node ./scripts/plugin-eval.js --help runs the CLI without a global install; npm link is the documented optional global command. - Install the full Codex plugin through the documented local marketplace instructions for natural-language skill invocation. ### Before you start - Node.js meeting the documented runtime requirement - Complete plugin files; a live Codex CLI session is needed only for benchmarks - Complete plugin checkout and its CLI dependencies - Codex CLI access only for live benchmarking ### Compatibility Codex plugin-eval plugin · Local Node.js CLI with the upstream-required runtime version ### Read target skill/plugin files and write local reports - Read target skill/plugin files and write local reports - Create benchmark configuration - Optional live Codex sessions in temporary workspaces ### Cost model MIT is declared in the plugin manifest; static inspection runs locally, while live benchmarks use the configured Codex service. 04 / FOLLOW THE EVIDENCE ## The source trail. Our notes are separate from the original resource. Check upstream before adopting a new version. - Skill definition at reviewed revision ↗ Checked 2026-09-21 https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/skills/evaluate-skill/SKILL.md Supports: summary, upstreamDescription, whySelected, bestFor, limitations, review, compatibility, access - Reviewed reference or helper: ../../references/chat-first-workflows.md ↗ Checked 2026-09-21 https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/references/chat-first-workflows.md Supports: review, limitations, access - Reviewed reference or helper: ../../src/core/analyze.js ↗ Checked 2026-09-21 https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/src/core/analyze.js Supports: review, limitations, access - Reviewed reference or helper: ../../src/cli.js ↗ Checked 2026-09-21 https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/src/cli.js Supports: review, limitations, access - Upstream license or licensing declaration ↗ Checked 2026-09-21 https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/.codex-plugin/plugin.json Supports: license - Publisher installation and project documentation ↗ Checked 2026-09-21 https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/README.md Supports: install, compatibility, access - Skills installer source formats and selective installation ↗ Checked 2026-09-21 https://github.com/vercel-labs/skills/blob/7407f3893ad4dceab546ac002c3ef806e4000c73/README.md Supports: install - Complete plugin setup and operational boundaries ↗ Checked 2026-09-21 https://github.com/openai/plugins/blob/1dc195897af4161d039b80d8471ec0a10c9bbc89/plugins/plugin-eval/README.md Supports: install, compatibility, access, review KEEP COMPARING ## Other approaches to consider. Related by category or shared topics. These are alternatives to inspect, not a measured quality order. [### Anthropic Skill Creator ↗ Develops skills through example tasks, baseline comparisons, human review and separate trigger-description evaluation.](/skills/anthropics-skill-creator/)[### Cloudflare Agents SDK ↗ Guides stateful Cloudflare agent applications through current SDK documentation, persistence, scheduling and approval-aware tool flows.](/skills/cloudflare-agents-sdk/) [AI Tools ↗](/tools/)[Skills ↗](/skills/)[Agents ↗](/agents/)[MCP Servers ↗](/mcp-servers/)