Start with the Process, Not the Tool

In this chapter

• Define the operational problem before selecting a product
• Compare AI and non-AI alternatives
• Write a pilot hypothesis that can be disproved

Name the friction precisely

An AI proposal should begin with an observed task, not a product demonstration. Harbor Pilot is a fictional operations team that rewrites incoming service requests into internal summaries. Its problem is inconsistent handoffs: reviewers must reopen the original ticket to recover missing details. 'Become AI-powered' is not an operational problem. 'Reduce summary preparation time while preserving all required facts' is specific enough to test and narrow enough to reverse.

Map the current process as intake, read, summarize, review, and route. Identify who performs each step and what evidence they use. A small template may solve missing fields without generation. A rules-based form may enforce required values more reliably. Keep those alternatives in the comparison. Selecting a simpler tool is a valid outcome when it meets the need with less cost or risk.

Use selection criteria, not a magic score

Compare three candidate tasks: internal summary drafting, refund approval, and public reply sending. Summary drafting has a visible source and can remain inside a review queue. Refund approval changes a financial outcome, and public replies affect customers directly. For this beginner pilot, choose internal drafting and explicitly exclude the other two. This is a scope decision for the fictional exercise, not a universal statement that those processes can never use AI.

Write the hypothesis before testing: 'On twenty synthetic tickets, assisted drafts will reduce median preparation time from the measured manual baseline, without increasing missing required facts.' Add a blocking condition: any draft that claims an action was completed when the source does not say so must be corrected before use. Define preparation time consistently, including revisions. Otherwise a quick first draft can hide a longer final review.

Watch for transferred work

The common failure is measuring only the author's time while shifting effort to the reviewer. Interview or observe both roles in a real authorized study; use role-play here. If drafting saves three minutes but checking adds four, the process has not saved a minute overall. Record that possibility in the selection memo instead of assuming automation creates value automatically.

Write the Harbor Pilot selection memo

• Describe the current five-step workflow and the reviewer handoff in a one-page memo.
• Compare a checklist, a form, and an AI draft assistant using evidence availability, review effort, and reversibility.
• Choose the internal summary task, write one measurable hypothesis, and list refund decisions and outgoing messages as exclusions.

Expected checks

• The memo describes a real task boundary rather than a slogan.
• A non-AI alternative receives a fair comparison.
• The success definition includes review time and a condition that could cause the pilot to stop.

Check your understanding

Which result would weaken the case for the assistant?

• It produces longer summaries.
• Drafting saves three minutes but review adds four minutes.
• Its interface looks different from the current tool.

Official tools & further reading

• NIST: AI Risk Management Framework overview