170 minutes already includes the 35 minutes spent fixing the draft

A weekly report used to take 40 minutes. The tool now returns a draft in 8 minutes. Those two times are part of a constructed example, not a measurement from a real team. The 32 minutes leaving the screen do not mean the organization saved 32 minutes. Some of that time may have moved into preparing the data, checking figures, editing sentences, or redoing the work when the output is wrong.

If you just finished a short pilot, the useful question is not how much faster is the machine? It is how the total effort to get an acceptable result has changed.

This piece picks up after you have designed the pilot. Assume one task is locked, you have a baseline — the time to do the same work before the tool — and you logged time for two weeks. Read that table before anyone talks about stopping, keeping the same scope, or expanding.

Evidence note: every figure below is a constructed example, built by the editors to show the arithmetic. It is not a standard measurement, not the result of running one particular model, and not a typical savings rate.

A pilot table can make the fast step look like the whole job

Suppose a small team tries the tool on weekly reports. Before the pilot, two weeks of the same volume took 240 minutes in total. That is the baseline.

After the tool was added, the team logged four kinds of time:

Work across the two weeks	Time
Prepare data and instructions	45 minutes
Produce the draft and steer the tool	35 minutes
Check figures and content	55 minutes
Fix errors and rewrite what was not good enough	35 minutes
Total effort	170 minutes

In the same period, the team recorded two cases where the output needed a substantial rewrite. You do not have to call that a failure of the tool. What matters is that the 35 minutes of rework are already inside the total.

Look only at the 35 minutes of drafting and the pilot looks extraordinarily fast. The fairer comparison is:

240 minutes of baseline minus 170 minutes of new total effort equals 70 minutes saved across the two weeks.

Those 70 minutes do not show that the tool will pay for itself over a long period. They answer a narrower question: in this scope, across these two weeks, recorded time was 70 minutes below the baseline.

Add the four groups before the number goes on a slide

If your table has machine runtime and nothing for preparation, checking, or repair, stop before you conclude. Add the four groups yourself. A short sheet is enough:

Question	Your team’s number
Baseline for the same volume of work	___
Preparing the input	___
Producing and steering the output	___
Checking	___
Fixing and redoing	___
New total effort	___
Gap against the baseline	___

Two things have to stay fixed: the same kind of output and a comparable bar for acceptable quality. If the old reports were checked carefully and the pilot versions went out with less checking, the time comparison is no longer fair.

Change one assumption and see whether the gap survives

A useful number should survive at least one simple boundary check. Keep the example above, and suppose repair time is 110 minutes instead of 35.

The new total is:

45 + 35 + 55 + 110 = 245 minutes.

The process then costs 5 minutes more than the baseline. That does not contradict the first table. It shows that rework is a variable that can erase the time advantage. This is a constructed case that changes one assumption on purpose. It is not an error observed in production.

You can press your own data the same way. If only one week had an unusual error, does the conclusion change? If you remove one very skilled person from the sample, does it change? If the volume of work rises, does checking time rise roughly with it?

The aim is not to make the pilot fail. The aim is to see which variable the conclusion depends on.

Keep 70 minutes from becoming a larger story

The NIST AI Risk Management Framework, from the U.S. National Institute of Standards and Technology, stresses that measurement stays tied to the context of use, the risk, and criteria you have already defined. Governance also needs clear roles, responsibilities, and monitoring. For generative AI, NIST’s profile for that class of tool keeps stressing evaluation, measurement, and testing of risks such as output that is wrong and still sounds convincing.

A two-week pilot should not be restated as “the tool saves the team 29% of its time every month” and then multiplied across a year. You do not yet know how learning the tool, a change in volume, hard cases, rare but serious errors, or the cost of oversight will move the result.

A conclusion that stays with the data can read like this:

Within the two weeks and the volume that were measured, the new process used 170 minutes against a baseline of 240 minutes. The 70-minute gap is still there after preparation, checking, and repair. More data is needed before generalizing to another team or deciding to expand.

If the pilot produced an error that affects a customer, sensitive data, compliance, or an important decision, a few dozen minutes of savings should not hide how serious that error is. Time is one part of the decision.

Three answers before you stop, hold the scope, or expand

You do not need more accounts yet. Write three answers:

1. Is the saving still there after rework is fully counted? If it is not, you already know what to improve before you expand.
2. Did quality stay at an acceptable level? Count the errors that had to be fixed, and separate serious errors from small wording fixes.
3. Is the result stable across weeks and across the people doing the work? One good week, or one very skilled user, is not yet the whole team.

The next step is not a prettier return-on-investment figure. Write the criteria first. Then decide to stop, keep the same scope, or expand from data you have checked.

Primary sources

1. Artificial Intelligence Risk Management Framework (AI RMF 1.0) — NIST — the source framework for governance, measurement in context, roles, and monitoring risk when you evaluate an AI system.
2. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1) — NIST — the generative-AI profile behind the caution about evaluation, testing, and output that is wrong but sounds convincing.