Skill Judge

/skill-judge

Evaluate a skill definition the user pastes against a multi-dimensional rubric and return scores, a grade, and prioritized fixes. Ask the user to paste the full SKILL.md (frontmatter and body); you cannot judge what you have not read. This skill judges existing drafts -- writing a new skill from scratch is skill-creator.

Core Philosophy

A skill is not a tutorial; it is a knowledge externalization mechanism. Its value is the knowledge delta:

Good Skill = Expert-only knowledge - What the model already knows

When a skill explains basics the model already knows, it is token waste -- context is a shared, scarce resource. Categorize every section as you read:

A good skill is >70% Expert content. Compute the rough E:A:R ratio as your first pass.

Evaluation Dimensions (100 points)

D1: Knowledge delta (25) -- the core dimension. Red flags (score <= 8): "what is X" sections, tutorials for standard operations, generic best practices ("write clean code"). Green flags: decision trees for non-obvious choices, trade-offs only experience teaches, edge cases from the field, "NEVER do X because [non-obvious reason]". Test each section: "does the model already know this?"

D2: Description and trigger quality (20) -- the description is the only thing the router sees; perfect content with a vague description is a skill that never runs. It must answer WHAT the skill does, WHEN to use it ("Use when..." plus specific scenarios), and contain searchable KEYWORDS. "Helps with document tasks" scores near zero. Mental test: write 5 prompts that should trigger it and 5 that should not -- does the description discriminate? Also check the fence: does it say what it is NOT for?

D3: Anti-pattern quality (15) -- half of expertise is knowing what not to do. Score high for specific NEVER lists with non-obvious reasons ("never validate after use because the fee calculation already ran"); score low for vague warnings ("be careful", "handle errors properly"). Test: would a domain expert read the list and say "yes, I learned that the hard way"?

D4: Thinking patterns vs mechanics (15) -- does it transfer how an expert THINKS ("before doing X, ask yourself...") plus domain-specific procedures the model would not know (non-obvious orderings, easy-to-miss critical steps)? Generic step-1-step-2 procedures for standard operations score low.

D5: Freedom calibration (10) -- specificity should match fragility. Creative tasks need principles and high freedom; fragile precise operations need exact low-freedom steps. Test: "if the model deviates here, what breaks?" High consequence demands low freedom. Rigid scripts on creative tasks and vague guidance on fragile ones are both mismatches.

D6: Structure and economy (15) -- is length proportionate to value? Clear routing for multi-path scenarios, no duplicated content, heavy reference material separated from the core execution path, examples that actually work. A 50-line skill can beat a 500-line skill; never let length impress you.

Evaluation Protocol

1. Knowledge delta scan: read the whole skill, tag sections [E]/[A]/[R], compute the ratio.
2. Description stress test: run the 5+5 trigger prompt mental test.
3. Score each dimension with quoted evidence -- cite the actual lines that earn or cost points. Unsupported scores are vibes.
4. Grade: A >= 90, B 80-89, C 70-79, D 60-69, F < 60.
5. Report as a chat artifact:

# Skill Evaluation: [name]
**Score**: X/100 (grade) | **Knowledge ratio** E:A:R = x:y:z
**Verdict**: [one sentence]

| Dimension | Score | Evidence |
|---|---|---|
[all six, each with a quoted line as evidence]

## Critical issues
[must-fix problems]

## Top 3 improvements
[ranked by impact, each with concrete rewrite guidance --
show the improved text, not just "make it better"]

Common Failure Patterns to Name

• The Tutorial: explains basics -- delete them, keep decisions and trade-offs
• The Dump: everything in one enormous file with no layering
• The Invisible Skill: great body, vague description -- never activates
• The Wrong Location: trigger info buried in the body where the router cannot see it
• The Vague Warning: "be careful" instead of specific NEVERs with reasons
• The Freedom Mismatch: scripts for creative work, vibes for fragile work

NEVER When Evaluating

• NEVER score high because it "looks professional" or is long
• NEVER forgive basics-explaining as "helpful context"
• NEVER skip mentally executing the decision trees -- do they lead to correct choices?
• NEVER give a score without quoting the evidence
• NEVER end without concrete rewrites for the top issues -- a grade without a fix path is useless

The Meta-Question

Always close with: would an expert in this domain read the skill and say "yes, this captures what took me years to learn"? If not, it is compressing what the model already knows -- garbage compression, whatever the grade breakdown says.

Fences

• Writing or scaffolding a new skill: use skill-creator.
• Designing a test/eval suite across many skills: use evaluation-methodology.

Usage

/skill-judge $ARGUMENTS