Answer Only What Is Supported

In this chapter

• Separate retrieval from answer support
• Reject unknown or unauthorized citations
• Define partial answers and abstention behavior

An answer is a set of checkable claims

Suppose Fieldnote retrieves a current passage saying the fictional AX-4 inspection interval is 30 days. An answer that adds replacement after six months contains a second claim that the passage does not support. A citation at the end of a paragraph can make both claims look justified. Structure the answer as individual claims paired with evidence identifiers, and make unsupported parts visible instead of filling them with plausible language.

A useful answer contract contains status, claims, evidence references, and limitations. Status can distinguish supported, partial, and insufficient evidence. These are application decisions, not promises of truth. Even a perfectly quoted source may be mistaken or outdated. For real equipment, authoritative operating instructions and qualified review remain necessary; this course uses imaginary data and does not provide servicing advice.

Validate mechanically, then evaluate semantically

The example accepts only an exact excerpt with an allowed citation. This deliberately narrow extractive mode is easy to test without an AI provider. A generative system would also need checks that each paraphrase is actually supported; matching identifiers alone is insufficient. Numeric values, units, negation, and conditions deserve explicit evaluation cases because a small wording change can substantially alter meaning.

evidence = {
    'e1': {'allowed': True, 'text': 'AX-4 inspection interval: 30 days.'},
    'e2': {'allowed': False, 'text': 'Supplier price: 90 credits.'},
}
def validate(claim):
    item = evidence.get(claim.get('citation'))
    return bool(item and item['allowed'] and claim.get('text') == item['text'])

claim = {'citation': 'e1', 'text': evidence['e1']['text']}
assert validate(claim)
assert not validate({'citation': 'e99', 'text': 'Invented procedure'})
assert not validate({'citation': 'e2', 'text': evidence['e2']['text']})
print({'status': 'supported', 'claims': [claim]})

Declining can still help

When evidence is missing, tell the reader which question could not be resolved and what kind of source would resolve it. For example: I found the inspection interval but no current storage-temperature specification. Do not claim that absence from your index means the specification does not exist. Distinguish an empty corpus, a permission restriction, a stale revision, and conflicting sources internally; expose only messages that do not reveal restricted material.

Retrieved documents can include malicious instructions, so keep them separated from application instructions and avoid automatically executing suggested tools. A validator is one defensive layer, not a proof against all prompt injection. If two allowed, current-looking sources conflict, return their dates and excerpts for review instead of selecting the answer that sounds more confident.

Build an honest answer packet

• Create supported, unknown-citation, restricted-citation, modified-number, and conflicting-source fixtures.
• Return a structured answer packet for each without calling an external model.
• Write reader-facing limitation messages that remain useful without revealing restricted document titles.

Expected checks

• Unknown and restricted citations are rejected.
• Changing 30 days to 300 days fails the extractive validator.
• The partial-answer message states what is known and what is missing rather than inventing a completion.

Check your understanding

A valid source identifier attached to a claim proves what?

• The claim is definitely entailed by the source
• Only that the identifier can be resolved; support still needs checking
• The source can be shared with everyone

Official tools & further reading

• OWASP prompt-injection prevention guidance