Session 3. Structured outputs — Wed 16 Sep

Quick quiz (ungraded)

Keeps it; an extra field is free information
A field nobody asked for is a field nobody validates. A contract that accepts anything is not a contract.
Drops it silently and returns the other four
Silent is the problem. You would never learn the model is drifting away from the shape you asked for.
Rejects the whole payload, naming the unknown key
Correct. The gate is set equality, so it reports missing=[] unknown=['source_url'] and nothing half-validated gets through.

A third attempt, with a stronger instruction
Retry until it works has no bound on cost or latency. A model that misread the instruction misreads it the fourth time too.
None, so the caller decides
That pushes the decision onto code with less context than the agent had, and every caller has to remember to check.
A ResearchAnswer with needs_human_review: true, empty citations, confidence 0.0
Correct. A typed refusal travels the same plumbing as a good answer and arrives where a person can see it.

Nothing; it passed the schema
Schema conformance is necessary, never sufficient. The citation is a doc id the retriever never produced.
The citation is stripped, confidence is capped, and the answer is flagged
Correct. The application verifies citations against what retrieval actually returned; the trace records the strip.
The parser raises AnswerParseError
The parser only knows the shape. It has never seen the retrieval result, so it cannot know that doc id is fabricated.

retrieve returns nothing, then a decision line, and no llm_call at all
Correct. The refusal happens before the model is asked, so it costs nothing and cannot hallucinate.
One llm_call, and the model says it does not know
That works too, and it spends a call to learn what retrieval already knew. Refuse before you spend.
Two llm_call lines, then a refusal
Two calls means the repair budget fired on a parse failure. That is a different failure from an unsupported question.

For logging; downstream code ignores it
It is the one signal that a flagged refusal produces. Ignoring it is how a refusal gets read as an answer.
So refusal is expressible inside the contract
Correct. A schema that cannot say I don't know pushes the model into inventing, because every accepted shape claims to know.
Because the provider's API requires it
The schema is yours. The provider returns text; every field in it is a choice you made and can test.