Sometimes you carry out research to confirm or test something:
Are people as happy with this thing as I think they are?
Sometimes you do it to get open feedback on something:
What exactly is it they’re happy (or fed up) about?
Then there are times when you want both to get a deeper understanding of something:
What are people happy (or fed up) about and do others feel the same? Or are people happy (or fed up) with this thing and why do they feel like that?
These times are when you’ll want to think about mixing your research methods: combining quantitative and qualitative research to uncover the numbers and the human stories behind them.
It’s more work and it takes more time, but the results can make all the difference.
Remind me, what’s quantitative and qualitative mean?
Quantitative and qualitative are words that don't exactly roll off the tongue. Luckily they are easier to understand than they are to say.
- Quantitative research is all about the numbers. It aims to quantify things. The cold, hard facts. If you’ve ever completed a survey filled with multiple choice questions and rating scales, that’s quantitative research. With the right questions and sample, it’ll produce data that helps you see the big picture.
- Qualitative research brings depth and detail in a way that quantitative research often doesn’t. You can explore what people really think and follow up on important points there and then. You’ll have seen this in practice if you’ve been at a focus group or taken part in a depth interview.
Why bother mixing your methods?
Mixing methods gives you a deeper understanding of a topic.
Quantitative research can help to identify or validate trends or behaviour, depending on when you carry out qualitative research. Qualitative research provides more context and reveals the “why” behind the trends and behaviours found in quantitative research.
For example, say you wanted to find out about people’s experience with a volunteering service.
Quantitative research approach
You survey 1000 volunteers and ask questions like: “How satisfied or dissatisfied are you with your volunteering experience?”
You can run analysis on the data and come up with conclusions, like: “75% of volunteers are satisfied with the experience ”.
Qualitative research approach
You interview 10 people and ask open questions like, “Tell me about your experience of volunteering with the service”, “What is the most positive aspect of volunteering?”, and “How could the volunteering service be improved?”.
You can then compare answers to find patterns and ask follow-up questions to clarify things. This will tell you what people like and don’t like. But the answers don’t speak for every volunteer.
To get the full picture, you’re going to want to bring the two types of research together.
Mixed methods approach
You interview people to get their thoughts on volunteering and find insights you never thought about before. You then run a survey to test these insights on a bigger scale.
Or, you could start with the survey to get an overall view of the volunteering service, then use interviews to get a better understanding of the reasons behind the trends.
This means you’re able to:
- Tell a more detailed story about the topic you’re researching
- Use the right methods for the right situation
- Cut down on unanswered questions at the end of your research, allowing you to focus on the most important ones
A mixed methods approach can be handy when:
- You’re researching a complex or nuanced topic (e.g. understanding the impact of the pandemic or social inequality on local communities)
- You’re evaluating something (e.g. the impact of your project or events on local communities)
- You’ve learnt from past research (i.e. previous findings have come up short on one side or the other)
To work out if combining methods is the way to go, ask yourself:
- Would we benefit from another kind of research?
- Would that research give us a better view of the question or problem?
For example, would the volunteering survey be more useful to us if we knew the reasons why people are satisfied or dissatisfied with the service?
You could add some questions to your survey to find out, but the feedback wouldn’t let you probe or ask follow-up questions like you can with qualitative research. Answers are also likely less detailed than you’d get from talking with someone.
Adding qualitative research to find out reasons first hand, would certainly help work out what’s good and what needs improving.
Designing mixed methods research
The approach you take to mixed methods research depends on whether you’re starting at a high level and working down, a low level and working up, or reacting to the needs of the project.
Starting with quantitative research
You start with a survey (quantitative) to get a broad idea of what’s happening (trends, themes and opportunities). Then you dig deeper into what the survey uncovered with interviews or observations (qualitative).
The survey picks out some key highlights and the interviews or observations let you pick out one or more things to focus on.
Starting with qualitative research
Going from small to big can be helpful when you’re working with a sensitive topic, small captive audience or something where you’re struggling to ask the right survey questions.
Here you can use context from interviews to create themes that then shape a larger survey or your approach to analysing existing data (e.g. operational databases or CRM systems).
Working reactively
Sometimes, rather than working top to bottom or bottom to top, you’ll need to work in both directions.
For example, you might run one method of research, explore the findings and, based on what they throw up, run more research in either direction to improve your insights.
Analysing mixed methods research
Broadly speaking, there are two ways to go about analysing the research. These pretty much mirror the way you design and collect data.
1. Top-down analysis
Use quantitative data you collect from surveys, analytics or service teams to guide the qualitative analysis by picking out some signposts.
So if, for example, surveys show that accessibility is a problem, you might start by looking at interviews where accessibility is quoted or given as an example.
2. Bottom-up analysis
Use qualitative research to pick out themes and create some theories, then clarify them with your wider survey results.
For example, if you found some discrete segments from your interviews, you can check how these segments are represented in your customer base.
If your data is already there, it can make more sense to start with quantitative data to create paths to dig down into with your qualitative analysis. This way, you’ll know a bit more about what you’re getting into so you can speed up your theme building.
If you’re working on an on-going project, you’ve got a bit more flexibility for one method to inform the next.
For example, 10 interviews might reveal 2 or 3 similar themes that you can explore at a higher level in survey data.
If you’re working to a tight deadline, analysing the data as it comes in might make your life a bit easier. By flagging standout surveys and comments from interviews you can clean the data as you go. This will help you hit the ground running when it’s time to make sense of it all.
Whichever way you go about it, combining approaches can improve your research by plugging gaps and covering any weaknesses in a single approach. The result is a more complete story. And the more you know, the easier it is to make confident decisions.
Need help with this stuff?
We love getting stuck into data and stories. Email us at hello@psresearch.co.uk or give us a call on 0750 6624 043.