How to measure training results after an AI prompting course

On-Off Group

October 2026

To measure training results, write down one before number for a real task, name the task people will use the new skill on, then check the same number two to four weeks later and look at what they produced. Feedback scores measure attendance, not behaviour.

This is for you if you have to report on training results and nobody set up the measurement before the course ran. The room was full. The scores were good. Now someone senior wants to know what changed.

The example here is an AI prompting course, but the method works for any training.

None of it needs a new platform or a survey tool. It needs one number written down before the session, and someone willing to go and look afterwards.

A full room and a good score only prove people showed up

On a normal Monday after a course, here is what exists: an attendance list, an average rating out of five, and a folder of certificates. That is enough for a report and not enough for a decision.

The scores are measuring the session. Was the trainer clear, was the food fine, did the exercises make sense. All useful for improving the next run. None of it tells you whether anyone opened an AI tool the following week.

We have run workshops for Philippine banks and insurers since 2015, and the sessions that changed how people worked were not always the ones with the best scores. Some of the highest rated sessions were the easy ones, where nobody had to try anything hard, and nothing changed afterwards.

So keep the feedback form. Just stop presenting it as evidence that the training worked.

Write down one before number, and do it before the session

The usual problem is not measurement skill. It is that nobody wrote anything down in advance, so there is nothing to compare against.

Pick one number that already means something to the business. Turnaround time on a policy amendment. Hours to produce the monthly branch report. Number of cases one person clears in a day. Time to get a first draft of a client email out.

Get the figure for the four weeks before the training. If nobody is counting it, count it by hand for a week. A rough figure taken honestly beats a precise figure invented afterwards.

Then say out loud what you expect to happen. "We think the monthly report drops from six hours to four." You are allowed to be wrong. You are not allowed to have no number, because that is how a six month programme ends in a slide pack and a shrug.

We do the same thing on delivery work. On a series of five day design sprints with a large insurer, the before number was time to market on customer facing web journeys: six months or more. After, those journeys were designed and tested with customers in two weeks and live in four. Completion rates on the new journeys rose 80 per cent and the old drop-off points disappeared. We can only say that because someone wrote down the 26 weeks before we started.

Name the first real task each person will use AI on

Course objectives tend to be written as capabilities. "Participants will be able to write effective prompts." Nobody can check that.

Instead, before the session, name the first real task each person will use AI on. Not a category. A task with a name and an owner. The complaint response drafts in the retail team. The weekly sales summary. The knowledge base answers the branch staff keep rewriting from scratch.

Give the trainer that list. A good facilitator will build the practice around it, and people will leave with a prompt for their own work rather than a prompt for a fictional company.

This also makes the check easy later. You are not asking whether skills improved. You are asking whether the complaint drafts got written differently, and whether the person doing them wants to keep going that way.

Go and look at the work two to four weeks later

Put a date in the diary for two weeks after the session and another for four. Then go and look at output, not opinions.

Ask to see the actual work. The prompts people saved. Three drafts produced with AI and three produced without. The report that used to take six hours. Sit with two or three people for twenty minutes each and watch them do the task. You will see within a few minutes whether the habit stuck, whether they are fighting the tool, or whether they quietly went back to the old way because the new way was slower on real data.

Then pull the same number you recorded before. Same definition, same source, same way of counting.

Expect a mixed answer. Usually a handful of people are away and running, most have used it once, and a few never opened it. That split is your finding, and it is worth reporting as it is. It tells you where the blocker is: permissions, workload, a manager who does not want the change, or a task that was never a good fit.

If the number is small, say the small number

The temptation is to lead the report with the attendance and the rating, then bury the behaviour. Do the opposite. Lead with the before number, the after number and how many people the change actually reached.

Small and honest is defensible. "Report time went from six hours to four and a half, for eleven of the thirty people trained" gives you a second conversation about the other nineteen. "Ninety-four per cent satisfaction" gives you nothing to work with next quarter.

It also keeps the scope sane. One task, one number, one team, then widen if it holds. We take the same approach to customer work, which is written up here: Instead of a platform project: ship one customer journey, measure it, then widen. Training works out the same way. Prove the change on one task and you have something to repeat. Try to measure whether everyone got better at AI and you end up back at the feedback form.

What to do this week

Pick one task per team that AI is supposed to help with, and name the person who owns it.
Get the before number for that task, even if you have to count it by hand for a week.
Write your expected after number down and share it with the sponsor now, not in the final report.
Book the two week and four week check in the diary before the training runs, and put it on the trainer's brief.
At the check, ask to see the work itself: saved prompts, drafts, the report. Not a self-assessment.

Questions people ask

What should the before number be?
Something already being counted on a real task, such as turnaround time on an onboarding case or how many hours a monthly report takes. If nothing is counted, spend a week counting it by hand before the training runs.

When should we check whether the training worked?
Two to four weeks after the session, on the first pieces of real work that came after it. Anything later gets mixed up with reorganisations, new systems and staff changes.

What if the number barely moved?
Say so, and check whether the task was the right one. A small honest number is still usable, and it usually points at a step in the process that the training was never going to fix.

Do certificates prove anything?
A certificate proves someone sat in a room and completed the exercises. It says nothing about whether they used the skill on Monday, so it is not evidence of a change.

On-Off Group trains teams, tests products with real customers, finds where a transformation has stalled and builds what gets it moving, for banks, insurers and enterprises in the Philippines, since 2015. How we help with training evaluation.