Generative AI & LLM Engineering Foundations

Learn to reason about model internals, adaptation budgets, evaluation, tool approval, and release operations through an offline support-category workbench.

What you'll be able to do

  • Explain tokenization, attention, and supervised loss with small calculations
  • Design leakage-resistant datasets and measured adaptation experiments
  • Separate model suggestions from authorized actions
  • Release a versioned model system with privacy-aware telemetry and rollback evidence

Before you start

  • Comfortable with Python, basic algebra, probability, and test design
  • Understand training versus inference and can read a confusion matrix
  • Python 3.11 or newer; optional model training requires a separately reviewed hardware and dependency plan

The curriculum

  1. From Tokens to Context — Free preview

    Use a tiny attention calculation to distinguish model representations from understanding, memory, and truth.

  2. Datasets That Can Disagree With You — Free preview

    Split related examples together, define loss targets deliberately, and prevent evaluation leakage.

  3. Adapt With a Measured Budget — Sign-in access

    Understand low-rank updates and quantization costs before choosing optional hardware-heavy experiments.

  4. Make Baselines Hard to Beat — Sign-in access

    Compare candidates using category-level errors, abstentions, and a frozen evaluation protocol.

  5. Proposals Before Actions — Free preview

    Keep model-generated suggestions separate from authorized tool execution and durable workflow state.

  6. Latency Has Components — Sign-in access

    Measure request stages and design caches that respect versions, identity, and changing access rules.

  7. Observe Without Exposing — Sign-in access

    Use telemetry and rollout gates to detect failures while limiting unnecessary collection of user content.

  8. Release Signal Bench — Sign-in access

    Integrate the offline classifier, review workflow, measurements, and compatible rollback into a demonstrable capstone.

Signal Bench

Build a local release workbench that classifies fictional support messages into categories, compares a baseline with a candidate, routes ambiguous cases to review, and records reproducible release decisions.

Deliverables

  • Synthetic grouped dataset and frozen evaluation set
  • Offline baseline classifier and candidate comparison
  • Reviewed action proposal state machine
  • Latency measurements, release manifest, and rollback demonstration

Review your work

  • Related examples do not cross training and evaluation groups
  • Category-level errors and abstentions are visible rather than hidden by aggregate accuracy
  • Model output cannot directly trigger an external action
  • Release gates and rollback are demonstrated using synthetic fixtures without paid calls