Skip to content
DZ Harness Hub

dz benchmark

dz benchmark runs a deterministic quality check on one or more skills. It evaluates 19 structural and content criteria (Layer-0 checks) and produces a Grade A–F report. For deeper AI-powered evaluation, use the /bto-test skill command for L1 (Haiku) and L2 (Sonnet panel) benchmarking.

The command takes a skill directory (the folder containing SKILL.md), not a bare skill name.

Basic Usage

Section titled “Basic Usage”
Terminal window
# Benchmark a single skill directory
dz benchmark .claude/skills/pr-review


# Benchmark all skill directories under a root
dz benchmark --all


# A/B compare two versions of a skill
dz benchmark .claude/skills/pr-review --compare ./pr-review-v2

The 19 L0 Checks

Section titled “The 19 L0 Checks”

L0 checks are deterministic — they do not require an AI call. They are split into 5 universal (U) checks and 14 skill-specific (S) checks. Every check either passes or fails:

# ID Check What It Validates
1 U1 SKILL.md exists A SKILL.md file is present in the directory
2 U2 valid UTF-8 The file is readable as valid UTF-8
3 U3 has headings Contains at least one Markdown heading (#–######)
4 U4 no excessive blanks No runs of 5+ consecutive blank lines
5 U5 size bounds (100B-50KB) File size is between 100 bytes and 50 KB
6 S1 frontmatter present Starts with a --- YAML frontmatter block
7 S2 name field Frontmatter has a name: field
8 S3 description field Frontmatter has a description: field
9 S4 trust_tier field Frontmatter has a trust_tier: field
10 S5 protocol/procedure section Has a procedural section (Protocol, Procedure, Steps, Workflow, Usage, …)
11 S6 output section or schema ref Has an ## Output section or references a schema_path
12 S7 schemas/output.json exists A schemas/output.json file exists
13 S8 scripts/validate-config.json exists A scripts/validate-config.json file exists
14 S9 when-to-use section Has a “When to use” / “Triggers” / “Use Cases” section
15 S10 no TODO/PLACEHOLDER markers No [TODO], [PLACEHOLDER], or [TBD] markers
16 S11 evals directory exists An evals/ directory exists
17 S12 content depth (>20 lines) The SKILL.md is more than 20 lines long
18 S13 examples section Has an Examples / Quick Start / Usage / Sample section
19 S14 anti-patterns or self-check Has an Anti-patterns, Self-check, Pitfalls, or Troubleshooting section

Grading

Section titled “Grading”

Grades are derived from the pass rate (gradeFromRate):

Grade Pass Rate Meaning
A ≥ 90% Production-ready
B ≥ 80% Minor issues
C ≥ 70% Needs improvement
D ≥ 60% Significant gaps
F < 60% Not usable

A Grade A skill is ready for production use. Most original DZ Harness skills target Grade A.

Example Output

Section titled “Example Output”
dz benchmark .claude/skills/pr-review


  SKILL: pr-review
  ─────────────────────────────────────
  ✓  U1   SKILL.md exists
  ✓  U2   valid UTF-8
  ✓  U3   has headings
  ✓  U4   no excessive blanks
  ✓  U5   size bounds (100B-50KB)        (8421 bytes)
  ✓  S1   frontmatter present
  ✓  S2   name field
  ✓  S3   description field
  ✓  S4   trust_tier field
  ✓  S5   protocol/procedure section
  ✓  S6   output section or schema ref
  ✓  S7   schemas/output.json exists
  ✓  S8   scripts/validate-config.json exists
  ✓  S9   when-to-use section
  ✓  S10  no TODO/PLACEHOLDER markers
  ✓  S11  evals directory exists
  ✓  S12  content depth (>20 lines)      (214 lines)
  ✓  S13  examples section
  ✓  S14  anti-patterns or self-check


  SCORE: 19/19 (100%)  →  Grade A

Batch Benchmarking

Section titled “Batch Benchmarking”
Terminal window
# Benchmark every skill directory under the default root
dz benchmark --all


# Sample output:
#  pr-review          A  19/19
#  security-audit     A  19/19
#  terraform          A  19/19
#  some-imported      C  14/19  (missing: S7, S8, S11, S14, S6)
#
#  Total: 4 skills   A: 3   B: 0   C: 1   D: 0   F: 0

A/B Comparison

Section titled “A/B Comparison”
Terminal window
# Benchmark two versions side by side
dz benchmark .claude/skills/pr-review --compare ./pr-review-v2


# Sample output:
#  Check                          pr-review   pr-review-v2
#  ──────────────────────────────────────────────────────
#  S13 examples section           PASS        PASS
#  S11 evals directory exists     FAIL        PASS   ← improved
#  S12 content depth (>20 lines)  PASS        PASS
#  ...
#  Score: 17/19                   19/19
#  Grade: B                       A           ← v2 wins

L1 and L2 Benchmarking with /bto-test

Section titled “L1 and L2 Benchmarking with /bto-test”

dz benchmark handles L0 only. For AI-powered evaluation:

Level Tool Description
L0 dz benchmark 19 deterministic structural checks
L1 /bto-test --level L1 Claude Haiku: 5 quality dimensions (clarity, completeness, examples, structure, anti-patterns)
L2 /bto-test --level L2 Claude Sonnet 3-judge panel: Expert, Critic, Auditor with weighted scores

Run dz create-skill --name <id> --bto to scaffold the eval templates, then invoke /bto-test in Claude Code.

Options

Section titled “Options”
Flag Description
--all Benchmark every skill directory under the root
--compare <dir> A/B compare against another skill directory
Section titled “Related Commands”