Single-review synthesis pilot: Gemini Flash, Flash-Lite, and Luna

Bottom line

Gemini 3.5 Flash-Lite at low thinking is the clear result of this small test. It validated all five rollups of the mistargeted case on the first attempt, preserved the differences among the five source-review treatments, and cost 0.63, or about $0.31 through Flex/Batch.

Gemini 3.6 Flash at medium also validated 5/5, but spent 11,623 thinking tokens to produce only 2,921 visible output tokens. It cost 6.15 standard or $3.07 through Flex/Batch.

GPT-5.6 Luna at medium on Flex is extraordinarily cheap and its three validated reports are the most detailed of the new candidates. It is not usable under the current evidence contract: only 3/5 first-pass outputs validated, and both failed treatments repeated the same non-verbatim quotation error on retry. This was not random API noise.

What was run

Every call received exactly one GPT-5.4-high evaluator review for the same resume — the mistargeted case. The five calls differed only in the evaluator prompt that had produced the source review:

1. anchored — control
2. anchored-mechanism — human-reading/mechanism passage
3. anchored-pressure — bias/leniency pressure passage
4. anchored-mixed — the integrated neutral prompt
5. anchored-negging — the integrated harsh prompt

The five reviews were never concatenated. The synthesizer saw no resume, job description, or candidate record. It received only the selected evaluator review, flattened into citeable prose fields plus score telemetry.

All calls used the same finding-synthesis system prompt, strict response schema, 8,192-token output allowance, and deterministic verifier. The verifier rejects the entire report unless every review_id and field exists and every quoted source is a byte-for-byte substring of that field.

Settings and cost

Gemini costs use billed output: visible response tokens plus thinking tokens. The local provider reports Gemini's candidatesTokenCount and thoughtsTokenCount separately. Luna's failed-call usage is unavailable because the current CLI discards the model result when quote validation raises, so its known cost is a lower bound. The estimate scales the mean of its three validated calls and should not be mistaken for measured five-call spend.

Token behavior

The regular Flash result is the opposite of token efficiency for this task. It used nearly four thinking tokens for every visible token and then returned the shortest reports. Flash-Lite produced more visible analysis, with no separately reported thinking tokens, for roughly one tenth the money.

What Flash-Lite preserved

The useful question is not the number of findings. It is whether the second stage faithfully exposes what each source review said without flattening the treatments into one generic answer.

• The control remained narrow: unused adoption/ROI evidence and incomplete keyword optimization.
• The mechanism review added the Microsoft-tooling gap, underused evidence for adoption/coaching/outcomes, and mildly overstated title/coaching framing.
• The pressure review retained omitted business-impact evidence, overformalization of independent work, and missing Microsoft/change terms.
• The integrated neutral review kept four distinct diagnoses: stack/change alignment, underused impact evidence, over-mature framing of self-directed work, and ambiguous employment labeling.
• The harsh review retained the additional governance-maturity and architecture-first-writing criticisms rather than replacing them with a generic low-quality verdict.

That is the desired second-stage behavior. It is reading and compressing the review prose; it is not averaging scores, voting across prompts, or pretending to independently judge the resume.

Flash-Lite is somewhat more willing than GPT-5.4 to label source-described gaps important, but in this one-review setup that does not distort cross-review agreement: every finding is necessarily found by one of one review instances. Severity should still be treated as presentation metadata, not a graph score.

What regular Flash added

Gemini 3.6 Flash reached essentially the same substantive center as Flash-Lite: missing adoption and ROI proof, Microsoft ecosystem mismatch, overformalized self-directed work, and—in the harsh treatment—governance and architecture-first writing concerns. It merged more material into fewer findings (15 total versus Flash-Lite's 16) and wrote more compressed diagnoses.

Nothing in the five reports demonstrates that its 11.6K thinking tokens bought better fidelity or a more useful human-facing artifact. On the mistargeted case, medium-thinking Flash is dominated by low-thinking Flash-Lite on cost and at least tied on quality.

Luna's failure mode

Luna's valid control, mechanism, and mixed-neutral reports are good. They are specific, preserve minor qualifications, and separate ATS gaps from underused business-impact evidence and from overstatement. Their five treatment-level finding counts would likely be higher than either Gemini model if all outputs passed.

But the evidence plumbing failed on anchored-pressure and anchored-negging:

• pressure: a citation to comments[1] was not a verbatim source substring;
• harsh: a citation to grounding.evidence was not a verbatim source substring.

Each call was retried once at identical settings. Each retry produced the same field-level validation failure. That makes the first-pass yield 60% and the retry recovery 0/2. A quote-repair pass or a source-span/index output format could probably make Luna operational, but that would be a different pipeline with an additional failure surface. Under the current fail-closed contract, Luna loses despite its negligible token price.

Decision

Use Flash-Lite, not regular Flash, for the next controlled expansion. The next test should keep the same five prompt treatments and add the two voice-pair cases already used in the Haiku pilot. That asks whether Flash-Lite's 5/5 result on the mistargeted case survives a case where the treatments differ on a narrow linguistic defect rather than on several overlapping targeting findings.

Do not turn this into a synthesizer score or a graph-quality score. The evidence supports a narrower claim: Flash-Lite is currently the cheapest tested model that can turn each mistargeted-case evaluator review into a valid, source-linked qualitative artifact without visibly flattening the review.

Artifacts

Each model/treatment directory contains the normalized synthesis-input.json, the validated finding-synthesis.json, a rendered finding-synthesis.md, and a per-call summary.json. Luna's rejected attempts contain error.json instead. The machine-readable aggregate is summary.json in this directory.