← Blog
Insight

Synthetic users are not users: what AI can and cannot do in research

In the last two years AI has changed the economics of user research. Transcription that took a day takes minutes. First-pass coding of twenty interviews happens over lunch. A repository you can ask questions of is finally realistic. We use all of this, and the work is better for it.

We are also seeing a worrying idea gain ground: that you can ask a language model to behave like your users and skip the research. You cannot, and the reasons matter.

What AI does well

Transcription and tidying. Accurate, fast and cheap. The only caution is consent: participants should know recordings are processed by a third-party model, and sensitive studies may need tools that keep data in-house.

First-pass coding. Asking a model to tag each statement with a theme gives a researcher a starting point in a fraction of the time. The researcher still reads the source. The model’s themes are suggestions, not findings.

Clustering open text. Thousands of survey comments or support tickets can be grouped into themes that a human then names and checks. This is genuinely new capability for small teams.

Drafting. Discussion guides, screeners, consent forms and the first draft of a report all start faster. They also all need editing by someone who knows the study.

Repositories. A searchable store of past research, with AI answering “what do we know about X?”, stops teams repeating studies. It only works if the underlying research is good and the answers cite sources.

What AI cannot do

Be a participant. A model trained on the internet produces plausible average opinions. Your users are not plausible averages. The whole value of research is the surprise: the workaround nobody predicted, the word that means something different in that community, the step that fails for someone with a screen reader. Synthetic users give you back your own assumptions in confident prose.

Decide what findings mean. A summary of ten interviews is not the same as having sat in them. Tone, hesitation, what people did rather than what they said, the moment a participant gave up: that is where insight lives, and it is not in the transcript.

Handle personal data casually. Interviews about health, money or benefits contain information that must be handled lawfully. “Paste it into the chatbot” is not a data protection strategy.

How we work

We run the same studies we always have, with real participants recruited to reflect the people who use the service, including those with accessibility needs. We use AI to get from recording to themes faster, to search what we already know and to draft. Every finding is traceable to a participant and a quote. Every report is written by the researcher who was in the room.

If your team wants to work this way, our AI-enabled research operations service sets up the tools, the guidance and the training. Faster research. Not fake research.

More from the blog

5 October 2026

Where AI belongs in a service: using journey maps and blueprints to find the right touchpoints

Start from the technology and you get a chatbot nobody asked for. Start from the journey and AI lands where it helps. Here is how we use…

Read article
1 October 2026

From idea to working AI prototype in a week: lessons from building MyStoryboarder and Compass

Proofs of concept used to take a quarter. With current models they take days. What we learned building our own AI product and a…

Read article
29 September 2026

Designing trust into AI services: what public sector teams need to get right

An AI feature in a public service is judged on trust before anything else. Transparency, fairness and the route to a human are design…

Read article

Want help applying this to your own site?

Book a free strategy call and we’ll look at your biggest opportunities together.

Book a call