How to set a baseline before a pilot starts

On-Off Group

October 2026

To set a baseline before a pilot, pick one number that matters to the business, write down exactly how it is counted and over what period, measure it for two to four weeks before anything changes, then measure the same way after.

If you have to tell your boss next month whether the onboarding pilot worked, start here. You have run the journey mapping. You have the new flow in front of branch staff. What you do not have is a before number, so the answer to "did it work" is a shrug and a story.

Setting a baseline before a pilot is mostly waiting: a short session to agree the number, then a few weeks of counting. That is the whole cost. Without it, you end up handing over a slide pack with no before number, and nobody can say whether the pilot did anything.

Five steps, in order.

Pick one number your boss already cares about

What usually happens: the team picks five metrics because nobody can agree, measures none of them properly, then reports whichever one moved.

Pick one. It should be something your boss mentions without being prompted. Turnaround time from submission to decision, or how many applications branch staff have to rekey.

With one number you can answer the question in the meeting without opening a spreadsheet.

A large insurer we worked with used two: how long it took to get a new customer-facing web flow live, and how many customers who started it finished it. Both were things the business already complained about. Nobody had to be convinced they mattered. So when the numbers came back, nobody argued about whether they were the right ones.

If you cannot name one number, the pilot does not have a goal yet. Sort that out before you build anything.

Write down exactly how it is counted

Most of the arguing later is about the definition, not the numbers. Turnaround time sounds obvious until three people count it three ways. Does the clock start when the customer hits submit or when the case lands in the queue? Does it stop at approval or at release of funds? And what about cases returned for missing documents?

Write the answers down in a paragraph. It does not need to be a policy document. Then show it to whoever will be asked to reproduce the number in three months and get them to agree.

This is the step that saves you later. When the after number looks good, somebody will say the definition changed. If the definition is written down and dated, you can settle it on the spot.

Keep the file somewhere other people can find it. Baselines get lost when the person who made them goes on leave.

Measure the current process for two to four weeks

Nothing changes yet. You are timing what already happens.

If the data is in a system, pull it. If it is not, count by hand. Pull a sample of cases from the queue, time each one from submission to decision, and record the date and who pulled them. A hand-counted spreadsheet is a real baseline, and it holds up when someone questions it.

Cover a normal cycle. If month end is busier, include a month end. If one branch behaves differently, note which branches are in the sample.

While you are in there, watch some of it. Sit with branch staff for an afternoon. The delay is often not where you thought it was. That is also the moment to check the flow is worth automating at all, because putting AI on a broken journey just makes the failure faster.

Note the conditions around the number

A baseline with no context gets picked apart. Write down what else was true during the before period. Volumes. Headcount on the team. Any system outage. A promo that pushed applications up.

This matters because something always changes that has nothing to do with your pilot. A branch closes. A competitor drops their rates. If you recorded the conditions, you can say so honestly and still make your case. If you did not, any improvement is easy for someone else to explain away.

A small number you can explain is easier to defend than a broad claim about efficiency. If turnaround time went from nine days to seven, say nine to seven, and say volumes were flat. That is more convincing than a claim about improved efficiency.

Then change one thing and measure the same way

Run the pilot. When it has been live long enough to cover the same kind of period, pull the number again using the paragraph you wrote.

The insurer's design sprints gave a clean comparison because both numbers existed. Web journeys that used to take six months or more were designed, tested with customers and live in four weeks. Twenty-six weeks down to four. Completion rates on the new flows rose 80 percent, measured against the old flows, and the old drop-off points went away. The client measured it themselves.

Four weeks only means something to your boss because the six months was written down first. It only lands because the six months was written down first.

The same applies to an AI feature you are still arguing about in scope meetings. Rather than argue about projected benefit, build a rough working version and watch someone use it, then measure that against what people do today.

What to do this week

Name the one number, and check your boss would recognise it without explanation.
Write the counting rule in a paragraph, date it, and get one other person to agree with it.
Start the before period. Pull the data or count forty cases by hand, whichever is faster.
Write down the conditions: volumes, headcount, anything unusual happening this month.
Put the after measurement in the calendar now, with the same counting rule attached. More of this work, and the projects behind it, are at onoffgroup.com.

Questions people ask

How long should the before period be?
Long enough to cover a normal cycle of work, usually two to four weeks. If volumes swing between the start and end of the month, cover a full month so the before and after periods look alike.

What if the data is messy or nobody owns it?
Count by hand. Forty cases pulled from the queue, timed from submission to decision, gives you a usable before number, as long as you write down how you picked them and repeat the same method later.

Can you set a baseline after the pilot has already started?
You can sometimes reconstruct it from old records, but the number will be argued with. If the pilot is only days old, pause the rollout for a couple of weeks and measure the current process first.

What number should a customer journey pilot use?
Usually turnaround time from the customer's first submission to the decision, plus the share of people who finish the flow. Both are things a customer feels, and both can be counted the same way before and after.

On-Off Group trains teams, tests products with real customers, finds where a transformation has stalled and builds what gets it moving, for banks, insurers and enterprises in the Philippines, since 2015. How we help with baselines and measurement.