Mixed-Prompt Qualitative Rollup

This is one narrow second-stage test of the résumé evaluator. It deliberately throws away most of the matrix: no Gemini, no negging prompt, no prompt ensemble, and no attempt to average the scores into a graph-quality number.

For each of the sixteen calibration cases, the input is exactly two complete anchored-mixed reviews:

• GPT-5.4 at high reasoning effort; and
• Claude Opus 4.6 with adaptive thinking at medium effort.

A separate GPT-5.4 medium call reads those two reviews and compiles their prose into source-cited findings and disagreements. Numeric judgments travel with the reviews as provenance, but they do not decide which findings survive. The synthesizer is explicitly instructed that a high score cannot cancel a problem described in the rationale or evidence.

All sixteen syntheses passed strict citation validation. The run produced a qualitative report for every case from 32 source reviews, at a total synthesis cost of about $0.61. The complete validated artifacts are preserved in resume-lambda/test_results/finding-synthesis/mixed-gpt54-opus46-16case-20260812/.

What this test says

The two-review rollup is useful. It turns two long evaluator responses into a readable issue list, preserves findings raised by only one judge, and puts real disagreements next to their verbatim evidence. It cleanly distinguishes the truncated floor-check case and the grounding-fabricated case, and it retains useful calibration findings in the pitch pair.

It does not solve perception. Neither source judge names the injected imperative voice, inline bolding, or buried-lead manipulation, so the second stage cannot recover them. The reports for those pairs mostly discuss other targeting and grounding issues. Collection can preserve a diagnosis; it cannot manufacture one that never appeared in either review.

The mistargeted case remains the most informative. The rollup correctly preserves the underemphasis on coaching, adoption, change management, and business-case evidence. It still leads with missing Microsoft/Copilot terminology and never states the full human diagnosis as sharply as it should: the body argues for an AI engineer while the job requires an embedded business partner. The second stage is therefore better at presenting the reviews than the scores are, but it inherits the reviews' framing and priority errors.

The practical conclusion is specific: this is a good human-facing triage artifact and a bad unattended graph score. For a graph comparison, read the findings and their sources, verify them against the résumé and candidate record, and compare the verified issue sets. Do not count findings, average severities, or treat two-model agreement as truth.

Individual case pages

The complete rollups are also available as sixteen standalone case pages. Each page has a stable URL, full findings and disagreements, verbatim source quotations, previous/next navigation, and the two original score rows collapsed at the bottom.

All case summaries

The table contains the stage-two bottom line for every case. Select a case below it to inspect the complete findings, disagreements, citations, and the original score telemetry.

Read one case

Findings

Disagreements

Original score telemetry

These values are included to make the test auditable. They are not inputs to a mean, vote, severity calculation, or graph-level score.

Reproduction

The source reviews are the anchored-mixed GPT-5.4 high and Opus 4.6 medium records filtered from:

test_results/evaluator-batches/anchored-mixed-negging-16case-3judges-20260811/results.jsonl

They were synthesized with GPT-5.4 at medium reasoning effort and a 16,384-token completion budget. Each generated finding and disagreement had to cite an exact review ID, field, and verbatim substring; any invalid citation would have failed that case. The run completed 16/16 cases.

.rollup-summary,
.finding-card,
.disagreement-card {
  border: 1px solid color-mix(in srgb, currentColor 18%, transparent);
  border-radius: 0.65rem;
  padding: 1rem 1.1rem;
  background: color-mix(in srgb, var(--theme-background, white) 96%, currentColor);
}

.rollup-summary h3,
.finding-card h4,
.disagreement-card h4 {
  margin-top: 0;
}

.finding-list {
  display: grid;
  gap: 0.85rem;
}

.finding-card header {
  display: flex;
  gap: 0.5rem;
  align-items: center;
  margin-bottom: 0.65rem;
}

.severity,
.dimension {
  border-radius: 999px;
  padding: 0.15rem 0.55rem;
  font-size: 0.75rem;
  font-weight: 650;
  letter-spacing: 0.025em;
  text-transform: uppercase;
}

.severity-blocking { background: #b91c1c; color: white; }
.severity-important { background: #b45309; color: white; }
.severity-minor { background: #475569; color: white; }
.dimension { border: 1px solid color-mix(in srgb, currentColor 28%, transparent); }

.finding-card blockquote,
.disagreement-card blockquote {
  margin: 0.8rem 0;
}

.finding-card blockquote p,
.disagreement-card blockquote p {
  margin-bottom: 0.35rem;
}

.finding-card footer,
.disagreement-card footer {
  font-size: 0.8rem;
  opacity: 0.75;
}