How to plan and run a diary study: a definitive guide

What is a diary study?

A diary study is a qualitative research method where research participants record what they think, feel, or do with respect to any research topic over a period of days or weeks. It's a longitudinal approach that gives participants the ability to capture what they do in the context of everyday life using videos, photos, screen recordings, or texts through a mobile app and share that with you over the course of the study.

The method is asynchronous. Unlike a focus group or interview where everyone is in the same room (or on the same call) at the same time, diary study participants record on their own schedule, in their own environment. That means researchers can observe repetitive behaviours, daily routines, and habits that a one-off conversation would miss entirely.

Diary studies aren't new. Researchers have been using paper-and-pencil diaries for decades. Like field studies and contextual inquiries, diary studies are a type of context method — they help you understand users in their real-life environment. What's changed is the technology. Mobile diary study apps mean participants can record a video while they're unpacking a product, snap a photo of their fridge, or screen-record how they shop online. Everything uploads to a dashboard where you can watch, comment, probe, and analyse without leaving your desk.

Why are diary studies so effective?

They work because you see what people actually do, as they do it, before they've had time to forget or rationalise. Memory is unreliable. We all construct neat narratives after the fact. Diary studies bypass that entirely.

You capture real behaviour, not recalled behaviour

Recall bias is a systematic error that occurs when participants don't remember previous events accurately — the accuracy of memories is influenced by subsequent events, the passage of time, and how the question is framed. It's a well-documented problem in any research that relies on self-reporting. In a focus group, someone tells you what they had for breakfast last Tuesday. In a diary study, they show you their kitchen at 7am. Over seven days, you see what they really eat every morning, and that repetition is where the truth lives. A single interview gives you a snapshot. A diary study proves patterns.

The researcher effect disappears

When a researcher is physically present, or even on a video call, people behave differently. This is known as the observer bias or Hawthorne effect — people modify their behaviour when they know they're being watched. They're more polite, more guarded, more likely to say what they think you want to hear. Diary studies are private and one-to-one. Nobody else is watching. Participants share from their own space, on their own terms, and the honesty shows, especially on sensitive topics.

Context reveals what words can't

You don't just hear about someone's shopping routine. You see the competing brands on the shelf, the cluttered kitchen counter, the notification that interrupts them mid-purchase. Context fills in details that people forget to mention, even in a good interview.

They work for sensitive topics

When the subject is money, health, personal habits, or anything people don't like discussing openly, diary studies give participants a safe, private space to share at their own pace. Focus groups are particularly poor for sensitive research because group dynamics and social desirability bias suppress honest answers.

How do diary studies compare to focus groups and interviews?

Diary studies, focus groups, and in-depth interviews each have their place. The question is which method gives you the data you need for a given research objective.

Focus groups are good for exploring reactions, generating ideas, and observing group dynamics. But participants are out of context, influenced by each other, and relying on memory. In-depth interviews give you depth and rapport, but they're still synchronous, still rely on recall, and you only get what the participant tells you in that window.

Diary studies do something different. They capture what people actually do, in their own environment, over time. You trade the spontaneous back-and-forth of a live conversation for something more durable: observed behaviour, repeated over days or weeks, with the ability to probe asynchronously.

Many research teams don't choose one or the other. Diary studies work well as pre-tasking before IDIs or group discussions (participants arrive having already reflected on the topic, and you have real-world footage to reference), or as post-interview follow-up (bridging the gap between what people say in a discussion and what they actually do afterwards).

What are the most common use cases for diary studies?

Diary studies work in pretty much any sector, at any stage of a project. They make up around half the projects on our platform. Here are the use cases we see most often.

Consumer behaviour and shopper research

Follow participants as they shop, cook, consume, and make purchasing decisions over the course of a week or more. Diary tasks can focus on purchase triggers, brand preferences, consumption occasions, or demand spaces. Screen recording lets you see exactly how someone navigates an online store while hearing what they're thinking.

UX discovery and product design

Watch real users interact with your product in their actual environment. Someone using your checkout flow on their sofa behaves differently than they would in a usability lab. Diary studies are well suited to the discovery phase of UX projects, where the goal is understanding user needs and identifying problems before jumping to solutions. They also help you discover or validate user personas, since personas are dynamic and diary studies can reveal how behaviours change over time.

Healthcare and patient experience

Understand patient experiences, treatment journeys, and health behaviours where they actually happen: at home, at the pharmacy, during daily routines. Diary studies give clinicians and researchers a view of the patient experience that clinic visits and retrospective surveys can't.

Financial services and sensitive topics

Money is something most people don't discuss openly. Focus groups and interviews are poor methodologies for financial research because participants either clam up or tell you what they think you want to hear. Diary studies give participants a private, comfortable space to share their real financial behaviours.

"I was absolutely amazed at how engaging, and personal and human this felt. Because it was over 3 weeks, I felt that I developed a different relationship than I would in 90 minutes. This was faster. This was cheaper. But it wasn't lower quality."
— Lucinda Craig, Qualitative Researcher, on Lloyds Bank's diary study with Indeemo

Academic and student experience research

Universities and academic researchers use diary studies to understand student experiences over time: adaptation to university life, social integration, health and wellbeing, study habits. A longitudinal approach picks up shifts in attitude and behaviour that a one-off survey at the end of term would miss completely.

Day-in-the-life and routine studies

The classic diary study application. Participants document their routines, habits, and experiences over days or weeks, giving you a view of behaviour over time that a single interview can't provide. Common examples include breakfast routines, commuting patterns, media consumption, beauty and grooming habits, and daily work routines.

How do you define your diary study research objectives?

As with any project, the first thing you need before launching a diary study is a set of research objectives. What are you trying to understand?

These should not be tied to the outcome of the study but rather to the questions you want answered. The goals should ensure coverage across the current challenges, blind spots, or unanswered questions in your brief.

If your diary study is at the very start of your innovation lifecycle, chances are you're not entirely sure what you should be asking. That's perfectly fine. In fact, the most successful discovery diary studies we've supported started with wide open tasks and iterated into the findings from there using agile, scheduled tasking.

Keep the goals simple and the tasks open-ended.

Some examples of research objectives:

• Understanding food or beverage occasions over the course of a week
• Tracking consumer beauty or grooming routines over a period of two weeks (and do they differ on weekends versus weekdays?)
• Keeping a diary of what consumers shop for on mobile over the course of a week (and how?)
• Understanding a week in the life of a sports fan (and specifically their match day experiences)
• Doing remote user research on small business owners and how they maintain their financial accounts
• Exploratory UX diary research on people's fitness routines and their use of apps and devices
• What does a week in the life of an electric vehicle owner really look like?
• Recording what it's like for a patient to live with a medical condition and the impact of diet or medication
• Understanding Gen-Z media consumption: what are they watching, on what platform, when, and why?

Setting goals gives you a stable framework for the study. It focuses the client and the team on the right objectives and ensures the tasking follows a coherent strategy.

If you're up for it, a truly agile diary study can start with the relatively open goal of getting a more contextual understanding of user behaviour. Embrace the unknown unknowns. Gather a few days' worth of data, quickly synthesise what's going on, and then add tasks as you go based on the trends you're seeing. Scheduled tasking is well suited to this agile approach, where you're tasking on the fly based on what you're learning.

How many participants do you need for a diary study?

There's no single answer. It depends on what you need to understand. But there are some rules of thumb.

If you're focused on one consumer segment or user persona, avoid having fewer than 10 participants. Any number below that makes your sample quite shallow, and any no-shows will have a big impact on the volume of data you collect.

On the upper end, once you go beyond 40 or 50 participants, you need to weigh up the incremental learning against the analysis time required. In reality, you'll be under time pressure, and with that much data, the chances are you won't have time to properly analyse everything.

If you're researching multiple user personas, have a minimum of six to eight per persona. Most studies land in the region of 10-30 participants where two to four personas or consumer segments are being researched.

How do you recruit and incentivise participants?

Recruiting the right participants can make or break your study. Screen rigorously and offer realistic incentives. No matter how loyal your super users are, you're still competing for their attention.

Finding the right people

Whether you're using Indeemo, a field agency, your own database, a client CRM, or going out onto social media, look for the persona or personas that fit the brief. Either brief this to the recruiter or have signals that help you discover the right persona in your own search.

We recommend a minimum of 8-10 participants per segment or persona. Quantity is not as important as quality, but don't go too shallow with your numbers either. Don't make up the numbers with just any participant. Get the right people.

With Indeemo, you can recruit participants from a global panel of 3 million+ in hours, or bring your own. Opt-in consent collection and incentive payments are handled through the platform.

Paying realistic incentives

Pay, pay well, and pay smart. We've supported hundreds of diary studies and we know what works. Poor incentives lead to roughly 20% registration rates. Appropriate incentives result in around 90% completion rates.

Sometimes, especially on longer longitudinal diary studies, you need to gamify it. Create a target. Offer a prize. You might look at paying a bonus for completing all tasks or a discretionary prize for the best uploads. The key is to treat incentives as a tool, not a necessary evil. NNGroup's guide to keeping participants engaged in diary studies recommends breaking incentives into milestone payments for longer studies, which reduces dropouts and keeps entries consistent. Our strategists can help you find the right incentive strategy for your study.

Being transparent with participants

Make sure participants are comfortable with the technology and clear on what's expected of them before the study starts. When you recruit, be explicit about:

• What the purpose of the study is
• What you need them to do
• How long it will last and how often they need to engage
• How much time it will take them to complete their tasks
• How much they'll get paid and when

Briefing participants up front, ideally by phone during the recruitment phase, ensures they buy in to your study. It also gives you the moral high ground when it comes to following up with anyone who's slow to get started.

How long should a diary study last?

Diary studies come in all shapes and sizes. Depending on what you need to understand, where, and when, a diary study can be as short as one day or as long as 12 months.

On average, most of the diary studies we support are between 7 and 10 days long. Depending on your use case, the window will vary:

Day-in-the-life: Allow for 3 days. One day to get everyone up and running on the app, one full focused day for diary tasking, and one day to allow stragglers to complete.

Week-in-the-life: Allow 9-10 days. Get participants registered and up and running, then allow a full clean week of data capture so you understand what a complete week looks like.

Long-term longitudinal: We've supported studies that have been ongoing for up to two years. On longer studies, screening and incentives are critical to ensure participants stay the course. Cadence is key. You won't be able to get participants to engage daily for very long, so you need to establish a rhythm and groove that works. Getting this right is complex, and we'd recommend talking to our team to discuss the specifics.

What are the best tasking strategies for diary studies?

Tasking is the part of the process where good studies are made or broken. Getting your tasks right matters more than almost anything else. We've done hundreds of diary studies, and our strategists will guide you through choosing the right strategy and optimising your discussion guide for mobile.

At a high level, you have four different tasking approaches.

Tips for writing good diary study tasks

Remember the medium. Participants will see your tasks on their phone. The screen is much smaller than the desktop you're typing on, so be mindful of scrolling. Keep it short and succinct. Ideally a task should never be longer than a tweet. Participants will have to remember the questions while recording a video or taking a photo, so stick to a maximum of three questions per task.

For example: "Record your Monday morning shaving routine. What time is it? Tell us about what razor you are using. Tell us about the products you are using."

Choose the right response type. Video, photo, text, screen recording. Each task should specify which format makes the most sense. Sometimes video is essential. Other times, a photo with a caption will do.

Use natural language. Write the way you'd talk to someone, not the way you'd write a questionnaire. You want participants to see you as a real person, not a faceless researcher in a white coat. One thing that works well: embed a selfie video of yourself in the introduction to your diary study. It builds rapport immediately and keeps things personal.

Task ideas for consumer and shopper research

If you're running a consumer diary study, here are some areas to explore:

• Purchase triggers: Ask participants to document every time they think about buying a product, noting what triggered the thought (an ad, a conversation, a calendar reminder) and whether they acted on it.
• Brand preferences: Have participants log their experiences as they shop, noting why they chose one brand over another and how satisfied they were.
• Consumption occasions: Ask participants to describe their preparations for key occasions (a dinner party, a weekend, a work event) and the products associated with them.
• Daily routines: A simple week-long diary of food and beverage consumption, noting the context (family dinner, desk lunch, late-night snack) and reasoning behind choices.

How do you engage with participants during a diary study?

What makes diary studies truly unique is the asynchronous method. Instead of relying on recall and a short interview window, diary studies let you see what participants repeatedly do over a longitudinal period.

This gives you more time to analyse real-time responses and start a dialogue with each participant through probing. Probing is where you, the researcher, ask follow-up questions and drill down on what someone has shared with you.

Going back to the "unknown unknowns," you have no idea what journey participants will take you on during a diary study. Keeping it interactive and conversational using comments and probes is the agile way to follow the data and dig deeper into what you're observing. Indeemo has real-time alerts and one-to-one commenting that makes this kind of in-the-moment probing easy.

• Ask why.
• Ask them to elaborate.
• Ask them to think about their response and qualify it.
• Be curious.

How do you analyse and present diary study data?

The key to a successful study is how you make sense of what you've collected and how you share it. Analysis used to be the bottleneck. With AI tools, it doesn't have to be.

Indeemo's dashboard includes:

• Automated video transcription and translation in 30+ languages
• Generative AI for summarisation, thematic analysis, and sentiment analysis
• Keyword analysis and keyword cloud for identifying trends
• Tagging for organising data
• Timeline, journey mapping, and collage visualisation tools
• Subtitled highlight reel creation for sharing with stakeholders

While reports are typically presented in PowerPoint or Keynote, supplementing your presentations with access to the raw data makes a real difference when it comes to building empathy. Letting your clients or internal stakeholders view participant uploads in an interactive dashboard changes the conversation. Observer logins that let clients see (but not engage directly with) participants are worth setting up.

Static slides sometimes struggle to convey the depth of a diary study. Presenting insights in-platform brings your participants and their context to life in a way that other methodologies can't. A 30-second video of a participant struggling with your checkout flow is worth more than a 30-page deck. It helps teams get closer to their customers and make better decisions, faster.

If you're using diary studies as part of a mixed-method approach, you can also import interviews or focus groups from Zoom, Microsoft Teams, or your computer into Indeemo, transcribe them in 30+ languages, and analyse them with generative AI alongside your diary study data.

How do diary studies fit into a mixed-method research approach?

Diary studies work well on their own. Many teams use them as their primary approach for discovery research, routine studies, or path-to-purchase projects. But they also combine well with other qualitative methods.

As pre-tasking before interviews or group discussions

Ask participants to complete mobile diary tasks before a deeper conversation. They arrive having already reflected on the topic, and you have real-world video and context to reference during the discussion. Research teams consistently tell us how much more productive their face-to-face sessions are when participants have already engaged through Indeemo beforehand.

As post-interview follow-up

After an interview, send participants follow-up tasks to document what they actually do in their daily lives. This bridges the gap between what people say in a discussion and what they do afterwards.

As a standalone longitudinal method

For research questions that require observing behaviour over time (consumption patterns, treatment journeys, user adaptation), diary studies are often the only method that can deliver what you need. A single interview or focus group can't show you what someone does every morning for a week.

Do you need to be a research expert to run a diary study?

No. Whether you're an experienced researcher or a brand team exploring diary studies for the first time, Indeemo can support you.

The app uses a social-networking-style UX that participants already know how to use. AI handles the heavy lifting on transcription, translation, and analysis. And if you need help, our team is there.

Use the platform independently if you have the expertise in-house. Or partner with our Catalyst team for study design, recruitment, moderation, analysis, or the full project. If you have research ambitions but not the capacity or expertise to fulfil them, we can lend a helping hand as and when you need it.

We've supported thousands of diary studies across every sector, from mapping consumer shopping journeys to understanding how patients live with chronic conditions to tracking how Gen-Z uses streaming platforms.