--- title: "Agent Platform Eval Flywheel: review, setup & limitations · Undominated.ai" canonical: https://undominated.ai/skills/google-agent-platform-eval-flywheel/ description: "Evaluates and iterates on Google Cloud Agent Platform models and agents, with a confirmation card before every billed evaluation run." --- # Agent Platform Eval Flywheel: review, setup & limitations · Undominated.ai > Evaluates and iterates on Google Cloud Agent Platform models and agents, with a confirmation card before every billed evaluation run. [← Explore all skills](/skills/) MODEL EVALUATION / Google # Agent Platform Eval Flywheel Evaluates and iterates on Google Cloud Agent Platform models and agents, with a confirmation card before every billed evaluation run. [See setup guidance ↓](#setup)[Original source ↗](https://github.com/google/skills/tree/4601752b28d601d54efb39b06a37488107fc6b1f/skills/cloud/agent-platform-eval-flywheel) SOURCE REVIEW Reviewed 2026-10-13 Evidence 4 linked sources Publisher Google Licence [Apache-2.0 ↗](https://github.com/google/skills/blob/4601752b28d601d54efb39b06a37488107fc6b1f/LICENSE) Revision 4601752b28d6 [Read what was—and wasn’t—checked ↓](#review) “Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology. Use when generating synthetic user scenarios, evaluating an agent or model, building an eval dataset, picking or writing evaluation metrics, analyzing failures, comparing results before and after a fix, or when guidance is needed on Agent Platform eval methodology — including dataset schema, LLM-as-judge scoring, and common failure causes. For fine-tuning, use agent-platform-tuning. For general production deployment, use agent-platform-deploy.” [Google · upstream description ↗](https://github.com/google/skills/blob/4601752b28d601d54efb39b06a37488107fc6b1f/skills/cloud/agent-platform-eval-flywheel/SKILL.md) Our analysis follows below. 01 / THE REASONING ## Why this made the selection. - Evaluation, inference and scenario-generation calls need their own Yes/No confirmation for every run, asked in one turn and executed only after the reply. - The file says never to fabricate scores or claim results not read from an actual result object, and to confirm all cases pass before declaring success. ### A good fit for - Running an Agent Platform evaluation with rubric metrics - Comparing results before and after a fix - Evaluate supplied responses without replacing them with a fresh inference pass. ### Weigh up before choosing - Setup tells the agent not to create a virtual environment and to pip install missing packages into the current environment. - The loss-clustering example creates a client in the global location, although the file otherwise says not to switch a user's location to global. Check the region on the confirmation card. - Five reference files and seven scripts were not opened. The README install command does not pin this revision. 02 / THE REVIEW RECORD ## What we actually inspected. Source review has boundaries. A clear record is more useful than a “safe” badge. ### Material inspected - skills/cloud/agent-platform-eval-flywheel/SKILL.md (frontmatter, when to use, safety and confirmation tiers, setup, failure analysis, iteration, proving your work, rules of engagement) - README.md (installation, skill list, plugins, licence) - LICENSE Apache-2.0 text - skills/cloud/agent-platform-eval-flywheel/SKILL.md - README.md - LICENSE ### Our findings - Helper scripts that only inspect, validate, parse or compare results are classed as read-only and need no confirmation. - After a fix, compare_results.py must show the target metric improved and no other metric regressed. - The evaluation instructions distinguish prompt-only datasets from datasets containing user-supplied responses, warning that another inference pass overwrites those responses. They require saving both JSON and HTML results. ### Not established by this review - The installer was not run, and the skill was not executed against a project or service. No evaluation was run and no Google Cloud project was contacted. - The references/ files and scripts/ were not opened. - Not executed: it was not installed or run. The review applies to the material and revision named here. A newer upstream release can change its behavior. 03 / PUT IT TO WORK ## Add a skill to your workflow. [Upstream setup instructions ↗](https://github.com/google/skills/blob/4601752b28d601d54efb39b06a37488107fc6b1f/README.md) DOCUMENTED COMMAND npx skills add google/skills Copy command ↗ Run in your terminal to choose your AI tools, project or global scope, and installation method. This short command follows the current upstream version; the review covers the revision shown above. - The README documents npx skills add google/skills and says to select specific skills, target agents and scope when prompted. Select agent-platform-eval-flywheel. - The command does not pin revision 4601752b28d601d54efb39b06a37488107fc6b1f. Compare the installed directory with that revision. - Approve each evaluation confirmation card only after checking project, location, model and dataset. ### Before you start - Node.js and npm for the documented Skills installer - A Google Cloud project with Agent Platform access, GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION, and Python for the scripts ### Compatibility Agent Skills-compatible hosts · Claude Code · Codex · Antigravity CLI · Google Cloud Agent Platform ### Implementation Python ### Google Cloud project evaluation runs - Local helper scripts run without confirmation; remote evaluation and inference calls require an explicit Yes for each run. - Writes evaluation datasets and result files locally and calls Agent Platform in the user's project. ### Cost model Source is Apache-2.0. Evaluation, judge and inference calls are billed to the user's Google Cloud project. Those charges were not measured. 04 / FOLLOW THE EVIDENCE ## The source trail. Our notes are separate from the original resource. Check upstream before adopting a new version. - [Skill definition ↗](https://github.com/google/skills/blob/4601752b28d601d54efb39b06a37488107fc6b1f/skills/cloud/agent-platform-eval-flywheel/SKILL.md) Checked 2026-10-13 https://github.com/google/skills/blob/4601752b28d601d54efb39b06a37488107fc6b1f/skills/cloud/agent-platform-eval-flywheel/SKILL.md Supports: summary, upstreamDescription, whySelected, limitations, access, review - [Install instructions and skill index ↗](https://github.com/google/skills/blob/4601752b28d601d54efb39b06a37488107fc6b1f/README.md) Checked 2026-10-13 https://github.com/google/skills/blob/4601752b28d601d54efb39b06a37488107fc6b1f/README.md Supports: install, compatibility - [Repository licence ↗](https://github.com/google/skills/blob/4601752b28d601d54efb39b06a37488107fc6b1f/LICENSE) Checked 2026-10-13 https://github.com/google/skills/blob/4601752b28d601d54efb39b06a37488107fc6b1f/LICENSE Supports: license - [Pinned continuation review W38 ↗](https://raw.githubusercontent.com/google/skills/4601752b28d601d54efb39b06a37488107fc6b1f/skills/cloud/agent-platform-eval-flywheel/SKILL.md) Checked 2026-10-15 https://raw.githubusercontent.com/google/skills/4601752b28d601d54efb39b06a37488107fc6b1f/skills/cloud/agent-platform-eval-flywheel/SKILL.md Supports: review.findings, bestFor KEEP COMPARING ## Other approaches to consider. Related by category or shared topics. These are alternatives to inspect, not a measured quality order. [### Arize Evaluator ↗ Creates and runs Arize LLM-as-judge and code evaluators through the ax CLI, with a rule against fabricating scores.](/skills/arize-ai-arize-evaluator/)[### LangSmith Evaluator ↗ Builds LangSmith offline and online evaluators, with rules for name-based deletes and judge model configuration.](/skills/langchain-ai-langsmith-evaluator/)[### Validate an Eval Scorer ↗ Checks an LLM judge or scorer against expert-reviewed labels, using agreement with uncertainty and severity-weighted errors.](/skills/braintrustdata-braintrust-validate-eval-scorer/) [AI Tools ↗](/tools/)[Skills ↗](/skills/)[Agents ↗](/agents/)[MCP Servers ↗](/mcp-servers/)[Workflows ↗](/workflows/) ## Continue your investigation - [Choose a toolkit](/tools/) - [Inspect execution hosts](/agents/) - [Check tool access](/mcp-servers/) - [Check project dependencies](/check-repo/)