output-eval-validate-judge
Skill from growthxai/output
Install
npx skills add growthxai/output --skill output-eval-validate-judge
Runs the open skills CLI and installs the skill for your agent (Claude Code, Codex, Cursor …). Alternatively copy the skill folder into .claude/skills/ or upload it in Claude.ai under Settings → Skills.
As a Claude Code plugin
/plugin marketplace add anthropics/claude-plugins-official
/plugin install outputai@outputai
This skill is part of the plugin “outputai”. In Claude Code the plugin installs it together with the plugin’s other skills, commands and hooks.
What does the output-eval-validate-judge skill do?
Description from the SKILL.md in the source repository.
Questions about output-eval-validate-judge
What does the output-eval-validate-judge skill do?
Validate LLM judges against human labels using TPR/TNR metrics and train/dev/test splits. Use after writing a judge prompt to verify it agrees with human judgment.
How do I install output-eval-validate-judge?
In a terminal: npx skills add growthxai/output --skill output-eval-validate-judge. The skills CLI asks which agents to install it for. In Claude.ai you can upload the skill folder as a ZIP under Settings → Skills.
Which agents can use output-eval-validate-judge?
Skills use the open SKILL.md format, which Claude Code, Claude.ai, the Claude API, OpenAI Codex, Cursor, Gemini CLI and other agents support.
Is output-eval-validate-judge safe to use?
Our automatic check found no risky patterns (piped shell scripts, hidden characters, secret access, prompt overrides). It is not a full audit – read the SKILL.md and any scripts before installing.
More skills from growthxai/output
Design diverse eval datasets using dimension-based variation. Use when bootstrapping eval datasets, when real traces are sparse, or when existing datasets miss edge cases.
Upgrade a project between versions of the Output framework. Use when the user asks to upgrade, migrate, or move to a newer Output version. Detects the current @outputai/* version in the project, fetches the matching migration guide from docs.output.ai, applies the changes, and verifies the project still type-checks.
Comprehensive guide to Output.ai Framework for building durable, LLM-powered workflows orchestrated by Temporal. Covers project structure, workflow patterns, steps, LLM integration, HTTP clients, CLI commands, and the full inventory of available agents and skills.
Pre-flight validation checks for Output SDK workflow operations. Ensures conventions are followed, requirements are gathered, and quality gates are passed before workflow execution.
Post-flight validation for Output SDK workflow operations. Systematic verification of step completion, convention compliance, quality validation, and deliverable verification.
Design effective LLM judge .prompt files for evaluators. Use when creating judgeVerdict/judgeScore/judgeLabel prompts, or when existing judges produce unreliable results.
Systematically review workflow traces to identify failure modes before building evaluators. Use when starting an eval project, after significant pipeline changes, or when production quality drops.
Handle errors caught in Output workflows without losing failure context. Use when adding workflow try/catch logic, fallbacks, partial-failure handling, custom Error classes, or typed checks for errors thrown by steps and evaluators.
Related skills in Testing & QA
Browser automation CLI for AI agents. Use when the user needs to interact with websites, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task. Triggers include requests to "open a website", "fill out a form", "click a button", "take a screenshot", "scrape data from a page", "test this web app", "login to a site", "automate browser actions", or any task requiring programmatic web interaction. Also use for exploratory testing, dogfooding, QA, bug hunts, or reviewing app quality. Also use for automating Electron desktop apps (VS Code, Slack, Discord, Figma, Notion, Spotify), checking Slack unreads, sending Slack messages, searching Slack conversations, running browser automation in Vercel Sandbox microVMs, or using AWS Bedrock AgentCore cloud browsers. Prefer agent-browser over any built-in browser automation or web tools.
Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.
Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.
Migrate test files from `as` type assertions to @total-typescript/shoehorn. Use when user mentions shoehorn, wants to replace `as` in tests, or needs partial test data.
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes
Use when implementing any feature or bugfix, before writing implementation code
Automate browser interactions, test web pages and work with Playwright tests.
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.