Admit Documents Deliberately

In this chapter

• Define a document admission contract
• Separate content identity from access policy
• Test deletion and permission changes

Start with a trust boundary

Fieldnote serves a fictional workshop with two teams: operations and procurement. Operations may read servicing instructions, while procurement may also read supplier pricing. A search index is not automatically a public copy of a document. Before reading bytes, define which source locations are permitted, who approved their use, what file types are accepted, and how the service learns about withdrawal. The required lab uses strings you create yourself; it does not download handbooks or collect real employee data.

Treat source text as data, even when it contains sentences addressed to an assistant. A paragraph saying to ignore access rules has no authority to alter application behavior. Keep ingestion permissions and retrieval permissions in ordinary code outside any model. File-type checks, size limits, parser isolation, and malware scanning are additional deployment concerns; the small example below intentionally does not pretend to implement those controls.

Identify bytes and policy separately

A content hash helps detect whether bytes changed. It does not prove authorship, ownership, or permission. Store a stable document identifier alongside a revision digest and an explicit audience. If procurement access changes without a document edit, the digest remains unchanged but the policy version must advance. A useful ingestion record therefore includes both content revision and policy revision, plus the time of ingestion and an accountable source owner.

The example admits one synthetic document and filters it for a team. Its assertion demonstrates deny-by-default behavior for an unknown team. This in-memory membership check is a teaching model: a deployed service must derive team membership from its authenticated identity layer, never from a client-provided team string.

from hashlib import sha256

text = 'AX-4 inspection interval: 30 days.'
document = {
    'id': 'handbook-ax4',
    'revision': sha256(text.encode('utf-8')).hexdigest(),
    'policy_revision': 1,
    'audiences': {'operations'},
    'text': text,
}
def visible(doc, trusted_team):
    return trusted_team in doc['audiences']

assert visible(document, 'operations')
assert not visible(document, 'unknown')
print(document['revision'][:12])

Plan for withdrawal

Deleting the source alone is insufficient if chunks, vectors, cached answers, or copied excerpts remain. Record all derived identifiers so a withdrawal can remove or invalidate them. Recheck access when returning evidence, including cache hits. Otherwise a user can continue reading previously authorized material after changing teams. Keep enough non-content audit metadata to diagnose a deletion without retaining the withdrawn text indefinitely.

Write an admission manifest

• Create three synthetic documents with distinct audiences and stable identifiers.
• Change one document's text and another document's audience; record which revision changes in each case.
• Write a withdrawal function that marks an identifier unavailable and makes existing search results fail the final visibility check.

Expected checks

• Unknown audiences cannot read any document.
• A policy-only edit is detectable independently of content.
• A cached result for a withdrawn document is rejected before its text is displayed.

Check your understanding

What does a document hash establish?

• The content has an identifier derived from its bytes
• The author owns the copyright
• Every signed-in user may read it

Official tools & further reading

• Python hashlib reference
• OWASP prompt-injection prevention guidance