Survey vs user research: record five people using your journey

On-Off Group

October 2026

A survey tells you how people felt about a journey they already finished. Recording five real people using it shows you where they stop, hesitate or call the branch. If your scores look fine and complaints keep arriving, watch the sessions.

If your customer satisfaction score sits at a respectable number every month, and the complaints inbox keeps filling up anyway, this is for you.

The survey vs user research question comes up whenever someone has to explain that gap to a boss. A survey asks people to rate something they have already finished. User research watches them do it. Those two things answer different questions, and only one of them tells you what to fix on Monday.

What follows is what changes when you record five real people using your journey instead of sending another form.

What the survey does not show you

A satisfaction survey reaches the people who finished. The ones who gave up on the ID upload, or rang the branch halfway through and had a staff member key it in for them, are not in your data. They either never got the form or ignored it.

So the score holds steady, and the complaints keep coming from somewhere your numbers cannot see.

The other problem is that a rating does not tell you which step went wrong. A three out of five on an onboarding flow does not tell you that people stall on the source of funds question because they do not know what counts. It tells you someone was mildly unhappy about something, at some point, for reasons they did not have room to type.

Branch staff usually know. Ask them where applications come back wrong and you will get a list in two minutes. That list is a starting point, not evidence, because the fix has to be shown to whoever signs it off.

What five recorded sessions give you instead

Five people, on video, using the live journey with their own phone and their own details. You watch where they stop, what they re-read, what they guess at.

By the third person the same blockers are repeating. From there the discussion is about a clip everyone has watched, not about whose opinion is right.

You leave with a short list of things to change, in order, with a reason next to each one. Not "improve the onboarding experience" but "the document upload fails silently on photos over a certain size and three of five people did not realise it had failed".

If you have not run one before, the mechanics are straightforward and we have written them up in How to run a usability test with five real users. Two weeks covers recruitment, the sessions and a highlight reel. Two weeks is enough for recruitment, sessions and a highlight reel.

Take the before number first

This is the part that gets skipped, and it is the part that decides whether anyone believes you later.

Before you change anything, write down one number about the journey as it stands today. Completion rate from start to submitted. Turnaround time from submission to approved. How many applications branch staff have to redo. Pick whichever you can get this week without a data request that takes a month.

Write the date next to it. Screenshot it.

We see plenty of teams who have done good work and cannot prove it. Six months of effort, a redesign that genuinely helped, and when the boss asks whether it worked the honest answer is a shrug, because nobody wrote down where it started. The research is wasted without the before number, and writing it down is the quickest part of the whole thing.

If the number turns out to be small after the fix, say so. A small number you can stand behind holds up when your boss pushes on it.

Ship one journey, then measure again

Once you have the recordings and the before number, fix one journey. Not the platform, not the whole app. One flow with a beginning and an end.

A series of five-day design sprints for a large insurer worked this way on customer-facing web journeys. Journeys that used to take six months or more were designed, tested with customers and live in four weeks. That is 26 weeks down to four. Completion rates on the new journeys rose 80 per cent, and the drop-off points that showed up in testing stopped happening. The client measured it, not us.

That works because the scope is small. One customer journey is small enough to record, change and measure again. A platform programme cannot, and by the time it lands nobody remembers what it was meant to improve. We have written more on that in Instead of a platform project: ship one customer journey, measure it, then widen.

If AI is on the list, do this first

You probably have a line in the plan about adding AI to something customer-facing. A chat assistant on the application form, document reading on the uploads, or something that summarises a case for branch staff.

Put any of that on a customer journey where three of five people already get stuck and they will get stuck faster. The customer still fails, just with a nicer interface in front of them, and now the failure is harder to trace because a model is in the middle of it.

The recordings tell you whether the problem is even one AI can help with. Sometimes it is: people cannot tell what document to send, and a reader that checks the upload on the spot would save two days of back and forth. Often it is not: the field is badly labelled, or the SMS code is slow to arrive and people give up waiting. No model fixes that.

More on the sequencing in Fix the customer journey before you put AI on it.

What to do this week

Write down one number for the journey you care about most, with today's date next to it.
Ask two branch staff where applications come back wrong. Keep the list.
Recruit five people who match your actual customers, not colleagues.
Record them using the live journey on their own phones, and keep the clips.
Pick the one blocker that appeared for three or more of them, and fix that first.

Questions people ask

How many people do we need to record?
Five is enough to find the big blockers in one journey. The same problems repeat quickly, and a sixth or seventh person usually shows you what you have already seen.

Do we still need the satisfaction survey?
Yes, as a trend line across months. Keep it for tracking, but stop using it to work out what is broken, because it cannot tell you where someone stopped.

What number should we record before a fix?
Pick one you already have or can count by hand: completion rate on the journey, turnaround time from submission to approval, or how many applications branch staff have to redo.

On-Off Group trains teams, tests products with real customers, finds where a transformation has stalled and builds what gets it moving, for banks, insurers and enterprises in the Philippines, since 2015. How we help with customer research.