← Résumé generator · Finding categories · Evaluator summaries

Evaluator scores vs human labels

All 16 cases, four prompts, and my recorded human labels in the last column. Each case has six dimension rows on the 0–2 scale.

↑ / ↓ means above / below the reference score or band, not better / worse evaluation. — means unavailable. Human labels marked † were previously derived from my prose. * marks an unscorable dimension or a still-provisional extrapolation; the case's label note explains which.

About the scores

These are the original evaluator judgments, not scores assigned by the summarizer. Changing the rollup model would not change them. The stored evaluator values are normalized to 0–1; this page restores their original 0–2 scale. Scores are shown individually, without averaging across dimensions or prompts.

Human labels come from the recorded, owner-ratified v2 labels, with subsequent explicit updates and confirmations stored in this page's human-scores.json. Explicit rulings and inclusive ranges are preserved. Still-provisional extrapolations are marked * with their assumptions recorded per dimension. The truncated case also uses * for dimensions that couldn't be scored. A full 0–2 range is unconstrained and is not treated as evaluator agreement. Recognizing a specific defect is a different question from agreeing with an integer score.

All sixteen historical cases are included, including the temporarily restored Skills-spam trap.

The four prompts match the finding-category grid: Base (anchored), Mechanism (anchored-mechanism), Criticism (anchored-pressure), and my integrated Combined passage (anchored-mixed). Source reviews remain available in the evaluation playground.