← Résumé generator · Evaluator scores vs human labels · Evaluator summaries

What the evaluators flag

Finding counts from summaries of individual evaluator reviews. Compare all 16 cases across four prompts; select the evaluator and the model that summarized its review.

Expand a cell's finding count to inspect its issues. Counts include readable summaries with unverified citations, marked below. Zero means no findings in that category; an unavailable summary is shown separately.

Reading the counts

Each cell represents one evaluator review summarized by the selected rollup model. Each finding counts once, regardless of severity or number of citations. Summary prose and disagreements are not counted as additional findings. These are counts of reported issues, not verified defects or résumé scores.

Categories use the summarizer's labels, with capitalization normalized. Labels outside Grounding, Writing, Argument, Keywords, Basics, and Overall appear under Other; expand the findings to see their original labels. A writing-related complaint may have been labeled Argument or another category, so these counts describe the summaries' categorization rather than a fresh classification of their content.

The four columns are Base (anchored), Mechanism (anchored-mechanism, human-reading passage), Criticism (anchored-pressure, evaluator-bias passage), and Combined (anchored-mixed, the integrated passage combining those themes). The archive also contains a separately concatenated treatment called anchored-combined; that is not the integrated prompt used here. The harsh/negging treatment is outside this comparison.

All 16 historical cases are included, including Skills-spam trap (Xcel), temporarily restored to this comparison. Its source-context concerns still apply. This page uses the existing single-review synthesis dataset; inclusion here does not change the active run manifest. For the original reviews and additional treatments, see the evaluation playground.