RAG Foundations: Retrieval-Augmented Generation

Build a permission-aware equipment handbook search service that can show exactly where an answer came from, measure retrieval failures, and decline unsupported questions.

What you'll be able to do

  • Build repeatable, permission-aware document ingestion
  • Compare lexical and vector retrieval with explicit evaluation evidence
  • Preserve citations across revisions, tables, and generated answers
  • Deliver a tested evidence service with an honest uncertainty policy

Before you start

  • Comfortable with Python functions, dictionaries, and tests
  • Basic understanding of HTTP and application authentication
  • Python 3.11 or newer; all required labs use the standard library and synthetic data

The curriculum

  1. Admit Documents Deliberately — Free preview

    Make ownership, permitted audiences, and document revisions part of ingestion before building a search index.

  2. Establish Search Baselines — Free preview

    Measure exact-word retrieval and a tiny semantic representation before introducing a larger search stack.

  3. Keep Evidence Attached — Sign-in access

    Choose chunk boundaries that preserve meaning and carry source coordinates through every transformation.

  4. Combine Rankings Without Hiding the Evidence — Sign-in access

    Fuse complementary retrieval results and account for duplicates, permissions, and missing evidence.

  5. Answer Only What Is Supported — Free preview

    Create answer contracts, validate citation identifiers, and make uncertainty useful to the reader.

  6. Read Tables With Their Context — Sign-in access

    Represent tables and visual evidence without losing units, headers, coordinates, or uncertainty.

  7. Measure Misses and Leaks — Sign-in access

    Evaluate retrieval, answer support, and access control separately with a reproducible question set.

  8. Ship Fieldnote With Evidence — Sign-in access

    Assemble a local capstone with versioned artifacts, reproducible tests, and a release decision grounded in measured behavior.

Fieldnote

Create a local equipment handbook evidence service for a fictional maintenance team. Search synthetic handbooks, restrict documents by team, and return traceable excerpts without claiming to provide operational safety advice.

Deliverables

  • Synthetic handbook corpus and versioned manifest
  • Lexical and hybrid retrieval baseline with an evaluation report
  • Citation-aware answer validator and abstention policy
  • Local command-line demonstration, tests, and release notes

Review your work

  • Unauthorized document text never appears in search results or citations
  • Every displayed factual answer has an allowed source and matching revision
  • Unsupported and conflicting questions return a clear limitation
  • An independent test set includes exact identifiers, paraphrases, stale revisions, and restricted sources