Full-matrix finding synthesis: Haiku vs Flash-Lite

Result

Gemini 3.5 Flash-Lite is the clear second-stage model for this pipeline. Across all 240 individual evaluator reviews, it produced 230 valid source-linked reports (95.8%) for 2.6633.

This is not another evaluator leaderboard. Each call reads one evaluator review and turns its prose into a compact, citeable issue list. It sees no resume, job description, or candidate data, and it does not average or vote on the original scores. The useful result is operational: Flash-Lite is a reliable, inexpensive reader of the reviews that already exist.

The source matrix is exactly five evaluator treatments × sixteen cases × three source judges. anchored-combined is excluded. Every one of the 480 synthesis calls was first-pass only; rejected quotations were preserved rather than repaired or retried.

What the full matrix adds

The small mistargeted-case pilot generalized. Flash-Lite validated at least 93.8% of every prompt-treatment slice and 92.5% or more for every source-judge slice. Haiku fell to 61.2% on the richer Opus reviews. Of its rejected outputs, 51 contain 78 altered quotations; five cite score telemetry six times even though only prose fields are valid citation targets; and one consumed all 8,192 output tokens without returning text. The score citations expose an interface ambiguity in the experiment rather than an invented source: scores were visible as audit telemetry but were not valid evidence fields.

The models are not secretly re-judging the resumes. Reviews from GPT-5.4 and Opus produce roughly two to four qualitative findings; the much more favorable Gemini reviews produce fewer than one. On the mistargeted case, both synthesizers return no findings for all five Gemini-source reviews while preserving the adoption, coaching, Microsoft-stack, self-directed-work, and architecture-first criticisms found in the GPT-5.4 and Opus prose.

That fidelity also exposes the limit. The sharper diagnosis on the mistargeted case—an AI-engineer document aimed at an embedded business-partner role—appears explicitly in only two valid Haiku reports and one Flash-Lite report. The models cannot recover a whole-document percept that the source review did not express. Likewise, neither names the injected imperative voice, inline bolding, or buried lead in the linguistic controls. By contrast, every valid output for the grounding-fabricated resume preserves the unsupported-claim diagnosis.

So the stage is doing what it should: making the evaluators' actual analysis legible, strengths and blind spots included. It is not adjudicating truth and it does not produce a graph score.

Settings

The measured prices use the standard on-demand rates: Haiku at 5/M output; Flash-Lite at 2.50/M output. A half-price batch run would have cost approximately 0.2318 respectively, but those are projections, not the service used here.

Exact second-stage prompt

You compile resume-evaluator outputs into a concise, actionable findings report.
You are not evaluating the resume yourself. You receive only evaluator reviews,
and every output claim must be traceable to those reviews.

Rules:
- Scan every supplied field, including summaries, rationales, evidence, and
  comments. A high numeric score does not cancel a problem described in prose.
- Preserve the union of substantive findings. Do not require majority agreement
  and do not discard a finding because only one review instance mentions it.
- Treat every review_id as a separate review instance. Instances may come from
  the same model with different settings, repeated samples, or intentionally
  perturbed inputs; do not assume they are independent providers.
- Merge findings only when they diagnose the same underlying problem. Similar
  subject matter is not enough. In particular, an omitted detail is not the
  same diagnosis as a whole-document targeting failure.
- Do not strengthen, repair, or reinterpret an evaluator's reasoning. If the
  reviews mention only local omissions, do not upgrade them into a broader
  strategic diagnosis.
- Agreement is not severity. Judge severity only from how materially the source
  reviews say the problem affects the resume.
- Surface meaningful contradictions under disagreements.
- Do not invent findings, evidence, recommendations, or source quotations.
- For every finding and disagreement, cite one or more exact source locations.
  `review_id` and `field` must match the input exactly, and `quote` must be a
  verbatim substring of that field.

Return JSON only in the required schema.

The response schema requires a summary, a list of findings with title, dimension, severity, diagnosis, and source citations, plus a disagreement list. The deterministic verifier rejects the entire response when a citation does not match the supplied review exactly.

Explore all 240 source reviews and 480 outputs

Source review

Second-stage outputs

How to use this

Use the explorer to read source-review prose and the two normalized outputs side by side. A valid citation means the source evaluator actually said the quoted thing. It does not mean the complaint is true, in scope, or correctly prioritized.

For generation-graph assessment, normalize the reviews with Flash-Lite, verify the resulting findings against the underlying artifacts, and compare graphs on the verified defects plus blinded same-input human preference. Do not count these findings or generated severities as a graph score.

.source-review,
.output-card {
  border: 1px solid color-mix(in srgb, currentColor 18%, transparent);
  border-radius: 0.7rem;
  padding: 1rem 1.1rem;
  background: color-mix(in srgb, var(--theme-background, white) 97%, currentColor);
}
.source-review > header,
.output-card > header {
  display: flex;
  justify-content: space-between;
  gap: 1rem;
  align-items: baseline;
  flex-wrap: wrap;
}
.source-review section {
  border-top: 1px solid color-mix(in srgb, currentColor 12%, transparent);
  padding-top: 0.5rem;
}
.source-review h4,
.finding h4,
.output-card h3 {
  margin: 0.5rem 0 0.3rem;
}
.output-grid {
  display: grid;
  grid-template-columns: repeat(auto-fit, minmax(min(100%, 28rem), 1fr));
  gap: 1rem;
  align-items: start;
}
.output-card.invalid { border-color: #b91c1c; }
.output-card.valid { border-color: #15803d; }
.status,
.severity {
  border-radius: 999px;
  padding: 0.15rem 0.5rem;
  font-size: 0.73rem;
  font-weight: 700;
  letter-spacing: 0.035em;
}
.output-card.invalid .status { background: #b91c1c; color: white; }
.output-card.valid .status { background: #15803d; color: white; }
.severity-blocking { background: #b91c1c; color: white; }
.severity-important { background: #b45309; color: white; }
.severity-minor { background: #475569; color: white; }
.finding {
  border-top: 1px solid color-mix(in srgb, currentColor 14%, transparent);
  margin-top: 0.9rem;
  padding-top: 0.8rem;
}
.finding blockquote { margin: 0.7rem 0; }
.raw-output {
  max-height: 34rem;
  overflow: auto;
  white-space: pre-wrap;
}