---
title: "Skill eval grader: review, role & definition · Undominated.ai"
canonical: https://undominated.ai/agents/anthropic-skill-eval-grader/
description: "Grades each expectation pass or fail from cited evidence, treats uncertainty as a failure, and writes grading.json."
---

# Skill eval grader: review, role & definition · Undominated.ai

> Grades each expectation pass or fail from cited evidence, treats uncertainty as a failure, and writes grading.json.

[← Explore all agents](/agents/)

EVALUATION / Anthropic

# Skill eval grader

Grades each expectation pass or fail from cited evidence, treats uncertainty as a failure, and writes grading.json.

 [Use this definition ↓](#setup)[Original source ↗](https://github.com/anthropics/claude-plugins-official/blob/b78ac49cdc6b3d7b61c4439470e311f4291265b1/plugins/skill-creator/skills/skill-creator/agents/grader.md)

SOURCE REVIEW

 Reviewed 2026-10-14
 Evidence 2 linked sources
 Publisher Anthropic
 Licence [Apache-2.0 ↗](https://github.com/anthropics/claude-plugins-official/blob/b78ac49cdc6b3d7b61c4439470e311f4291265b1/plugins/skill-creator/LICENSE)
 Revision b78ac49cdc6b
 [Read what was—and wasn’t—checked ↓](#review)

“Evaluate expectations against an execution transcript and outputs.”

 [Anthropic · upstream description ↗](https://github.com/anthropics/claude-plugins-official/blob/b78ac49cdc6b3d7b61c4439470e311f4291265b1/plugins/skill-creator/skills/skill-creator/agents/grader.md) Our analysis follows below.

01 / THE REASONING

## Why this made the selection.

 - The inputs are expectations, a transcript path, and an outputs directory, and each expectation is pass or fail.
- The grader is also told to flag an assertion that would pass for a clearly wrong output.

### A good fit for

 - Supply explicit expectations, a complete execution transcript and accessible output files with suitable inspection tools.
- Receive grading.json beside outputs, with evidence per expectation, verified claims, user notes and critique of weak assertions.

### Weigh up before choosing

 - This plain Markdown role has no native frontmatter or tool allowlist; provide it through an orchestrator with appropriate output inspection tools.
- Binary-document inspection and external claim verification depend on tools actually available in the session.
- Its JSON example contains illustrative counts/times; copy only supplied measured metrics and recompute aggregate verdict counts.

02 / THE REVIEW RECORD

## What we actually inspected.

Source review has boundaries. A clear record is more useful than a “safe” badge.

### Material inspected

 - plugins/skill-creator/skills/skill-creator/agents/grader.md
- plugins/skill-creator/LICENSE
- Definition reread at the preserved revision; scoped workflow and output-contract review on 2026-10-14

### Our findings

 - The file is titled Grader Agent and has no YAML frontmatter.
- Required prompt parameters are expectations, transcript_path, and outputs_dir.
- Each expectation is pass or fail, with no partial credit.
- The output includes evidence for the verdict, and eval_feedback only when the assertions have a clear gap.
- The grader checks actual relevant output files rather than relying only on executor descriptions, and treats unverifiable expectations as failures.
- It distinguishes copied executor metrics/timing from its own judgments and asks for assertion gaps that could let wrong outputs pass.

### Not established by this review

 - The agent was not executed against a live repository or host.

The review applies to the material and revision named here. A newer upstream release can change its behavior.

03 / PUT IT TO WORK

## Use the role in your project.

[Upstream setup instructions ↗](https://github.com/anthropics/claude-plugins-official/blob/b78ac49cdc6b3d7b61c4439470e311f4291265b1/plugins/skill-creator/skills/skill-creator/agents/grader.md)
 - Copy grader.md at the reviewed commit into the location your host uses for agent prompts.
- No install command is published. Installing the whole skill-creator plugin was not reviewed.

### Before you start

 - An execution transcript
- An outputs directory
- A list of expectations

### Compatibility

Hosts that can load a Markdown agent prompt

### Implementation

Markdown

### Reads a local transcript and writes grading.json beside the outputs

 - No tool allowlist is declared.
- The process says to read the transcript and the output files.

### Cost model

Apache-2.0. The file does not state a hosted-model price.

04 / FOLLOW THE EVIDENCE

## The source trail.

Our notes are separate from the original resource. Check upstream before adopting a new version.

 - [Grader prompt ↗](https://github.com/anthropics/claude-plugins-official/blob/b78ac49cdc6b3d7b61c4439470e311f4291265b1/plugins/skill-creator/skills/skill-creator/agents/grader.md) Checked 2026-10-14 https://github.com/anthropics/claude-plugins-official/blob/b78ac49cdc6b3d7b61c4439470e311f4291265b1/plugins/skill-creator/skills/skill-creator/agents/grader.md Supports: summary, whySelected, output format
- [Apache-2.0 licence ↗](https://github.com/anthropics/claude-plugins-official/blob/b78ac49cdc6b3d7b61c4439470e311f4291265b1/plugins/skill-creator/LICENSE) Checked 2026-10-14 https://github.com/anthropics/claude-plugins-official/blob/b78ac49cdc6b3d7b61c4439470e311f4291265b1/plugins/skill-creator/LICENSE Supports: Redistribution terms, Unfilled copyright appendix

KEEP COMPARING

## Other approaches to consider.

Related by category or shared topics. These are alternatives to inspect, not a measured quality order.

 [### Post-hoc skill analyzer ↗ Skill-creator agent that reads a finished blind comparison, then opens both skills and transcripts to say why the winner won.](/agents/anthropic-skill-analyzer/)[### Plugin skill reviewer ↗ Reviews a skill file for structure, description triggers and progressive disclosure.](/agents/anthropic-plugin-skill-reviewer/)[### Trojan Skill Hunter ↗ Reviews agent instructions, hooks and MCP configuration as untrusted text before installation.](/agents/github-trojan-skill-hunter/)

 [AI Tools ↗](/tools/)[Skills ↗](/skills/)[Agents ↗](/agents/)[MCP Servers ↗](/mcp-servers/)[Workflows ↗](/workflows/)

## Continue your investigation

 - [Find reusable instructions](/skills/)
- [Inspect connections](/mcp-servers/)
- [Build a toolkit](/tools/)
- [Choose the model](/compare/)
