advanced-evaluation

advanced-evaluation is an open-source workflow skill for Claude Code and compatible agents, published by sickn33. Its author describes it as: “This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bia…”. The project has 45k stars on GitHub and is available under the MIT license. Add it to your setup with `/plugin marketplace add sickn33/agentic-awesome-skills`.

What advanced-evaluation does

This skill covers production-grade techniques for evaluating LLM outputs using LLMs as judges. It synthesizes research from academic papers, industry practices, and practical implementation experience into actionable patterns for building reliable evaluation systems.

Installation

Add advanced-evaluation to your agent with:

/plugin marketplace add sickn33/agentic-awesome-skills

Always review a skill's source before installing it. This command comes from the skill's public repository; the linked repo is the source of truth for exact setup steps.

What's inside

The SKILL.md for advanced-evaluation is organised into these sections:

• When to Use
• Core Concepts
• The Evaluation Taxonomy
• The Bias Landscape
• Metric Selection Framework
• Evaluation Approaches
• Direct Scoring Implementation
• Pairwise Comparison Implementation
• Rubric Generation
• Practical Guidance
• Evaluation Pipeline Design
• Common Anti-Patterns

When to use it

Reach for advanced-evaluation when you want workflow help from your agent without writing the same instructions every session. Load the skill and the agent picks it up automatically for relevant tasks.

Strengths

• Clear MIT license — safe to read and adapt
• Ships in sickn33/agentic-awesome-skills, an established project with 45,095 GitHub stars
• Actively maintained (recent commits)

Topics

Frequently asked questions

What does advanced-evaluation do?

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

How do I install advanced-evaluation?

Run /plugin marketplace add sickn33/agentic-awesome-skills in your agent, then reload your skills. Review the source at https://github.com/sickn33/agentic-awesome-skills before installing.

Is advanced-evaluation free to use?

Yes. advanced-evaluation is free and open source under the MIT license, so you can read, run, and adapt it within that license's terms.

Where does advanced-evaluation come from?

advanced-evaluation ships inside sickn33/agentic-awesome-skills, a repository that contains 66 catalogued skills in total. The repository's 45,095 GitHub stars apply to that whole collection, not to this skill on its own.