It’s tricky to get concept validation right. In this article, we’ve outlined 6 common mistakes you might run into when doing concept validation and 6 tips for avoiding them.
Whether you’re a seasoned market researcher or just someone tasked with validating product ideas, it’s easy to get side-tracked in your search for a winning idea. Sometimes it happens because you’re juggling several projects at a time, and sometimes you simply don’t have enough expertise to assess ideas quickly. In this article, we’re going to cover a few common mistakes that happen when you do concept validation and our tips on how to resolve them quickly.
On the one hand, concept validation is a great way to test the viability of your idea. Many businesses use it to test their hypotheses about how their target audience will react to a new product, identifying those things that are likely to attract the right people to their idea.
However, there are many pitfalls that product marketers and researchers face along the way. And when this happens, the process might actually work against them. Projects might take much more time to be developed and costs associated with additional research and development (R&D) are likely to rise.
Why concept validation errors happen:
Lack of strategy
According to McKinsey, 84% of executives agree that innovation is important to growth strategy, but only 6% of them are satisfied with innovation performance. One of the reasons why so few are happy with the results is the lack of a strategy.
When you don’t take time to understand your market at the early stages, it’s harder to implement plans to avoid failure or stagnation. You need to step back and ask yourself, “How do we get to our end-point?”. This will guide you to understand why you’re doing your concept validation test, find the right approach to your research and ultimately create a minimum viable product (MVP) at the end of it.
Thinking too broadly or too narrowly
When you’re launching a new product, it’s tempting to ask your target market everything you want to know about them. The drawback of that is that you end up with a long survey that doesn't have any focus or direction. This results in a bad user experience (I mean, respondents will only answer so many questions!) and will make reporting and analytics a bit of a nightmare.
The flipside of the above approach is making your research far too narrow in focus. While it’s good to focus on what you want to achieve, you still want to make sure that you’re taking into account other possibilities.
For instance, if you are an energy drink company and want to learn which competitor brands your customers buy, don’t just limit yourself to energy drinks. Think first of other alternatives that your potential customers might consider - e.g. an energy bar, functional soda, or even just coffee. When you consider that your customers might not define your category as you do, your user research will be much more diverse, interesting, and inspiring.
Thinking only about the “now”
One of the common reasons people get side-tracked in their concept testing research is that they only focus on how their customers shop right now. While it’s good to have information about the current market trends, this type of research isn’t likely to help you in the long run. Customer needs change quickly when disruptors (businesses and startups with innovative ideas) go-to-market with something different. That’s why you always need to think a few steps ahead to make sure that you capture the unmet needs of your target audience
Let’s use the energy drink example again. If you want to forecast how your target market might shop in the future, you’ll want to think about why your customer shops for energy drinks in the real world. Think of innovative use cases and formats that aren’t that widely adopted or don’t exist yet - e.g. energy gums, caffeine pills or shots, natural sources of energy (e.g. herbs and plants), energy shampoos or creams… The list of new product ideas can be as creative as your imagination (and with a platform like Upsiide, you can test as many as you’d like against each other).
Once you give your respondents an option to choose between ideas that exist right now and those that you could develop in the future, you’ll get a true sense of the viability of the brand new innovation ideas.
Common mistakes in concept validation and how to avoid them:
1. Rejecting ideas because they didn’t immediately perform well
People like dividing things into good and bad, but this kind of approach doesn’t work in research. Once you’ve completed your concept validation test, you’ll see that some ideas performed better than others.
But if you take this info away and only focus on the ideas that did particularly well, you’re overlooking the fact that some of the ideas aren’t actually bad - they just weren’t as good as the ones that came out on top in their current form/description.
When you ignore those ideas, you throw away a whole pile of things that might have potential in the future. What if an idea didn’t perform well because it was too different from the norm, but will be accepted in the future? When Apple released the iPhone to be sold for $499, its competitors mocked it for its unreasonably high price. Now, the phone’s price has doubled and yet, millions of people are willing to upgrade their iPhones every year.
How to avoid this mistake:
Instead of dividing ideas into good and bad, spend time digging into the results to surface those ideas that have potential. Create a list of these ideas. Occasionally, when you’re looking for some inspiration for your next innovation, go back to that list and think of ways to improve the original idea. Once you’ve come up with an improvement, test it again against your in-market products.
Another approach could be to do more research into those concepts that didn’t perform as well. Look deeper and see if these ideas performed better with certain market segments. It might be a case that even though the idea didn’t do so well with the general population, a small group of respondents are very interested in it because they have a specific set of needs.
We’ve got a real example of this; when we tested different plant-based burger brands, The Very Good Butchers wasn’t the most popular brand. But when we dug through the data, we found that a small but passionate proportion of the market actually prefer The Very Good Butchers over other brands, meaning that The Very Good Butchers has an opportunity to tap into a niche but powerful segment of the market.
Read more about how we evaluated plant-based protein brands here.
2. Waiting until you’ve got full concepts to test
Oftentimes, when you think of testing product ideas, you assume that you need to already have some sort of prototype or design in place. Sometimes research teams have to wait for months before the first product prototype is ready to test with the market, which slows down the concept testing process. Now, imagine if you want to test a few products? The whole process might take years to evaluate.
But we did some research on research and found that you don’t actually need the full concepts to test their viability. Our recent studies on limited-time offers (LTOs) in the quick-service restaurant (QSR) space showed that you don’t have to have a visual or creative ad mocked up for testing.
We did 2 studies - one contained written LTO descriptions and one contained LTOs visualized with pictures in completed ads. When we compared the results, they were largely identical: for instance, a McDonald’s LTO description performed the same way as the complete ad that McDonald’s circulated everywhere. That’s how we discovered that you don’t always have to have a complete concept at hand if you already have a good description of it. Read more about this specific study here.
How to avoid this mistake:
Our discovery demonstrates that you don’t have to spend months and years developing new prototypes. Simply create a list of ideas, write down a few comprehensive descriptions for each and test them. This process will help run research earlier in your process and inevitably save you time, resources and money. There’s no use developing full-fledged product prototypes to test when you can get an early sense of a product’s potential to succeed with a description.
3. Considering the audience as a whole, not as a diverse set of people
A common mistake in concept validation leads us to the next one - thinking of your respondents as a group, not as individuals. When you validate ideas with hundreds and thousands of people, we sometimes forget that they are customers that have unique preferences.
Going back to The Very Good Butchers example, we found that there is a segment of people who are interested in brands that have an artisanal feeling to them - e.g. employ unique, hand-crafted packaging or use on-pack claims that position them as independent or handmade. While this group was smaller than the rest of the population, they’re responses indicated a considerable amount of loyalty to The Very Good Butchers and brands similar to it.
How to avoid this mistake:
Run a usage & attitudes study, create a few customer personas and understand their needs and wants. We recommend using the Jobs-to-be-done (JTBD) framework to understand why customers use your product. We explored JTBD in our previous article and explain how knowing your customer’s “job” can help you develop products that people will buy.
Once you get to know your target audience better, you can validate your product ideas and check how different audience groups react to them.
4. Forgetting to iterate
Across all our articles on concept validation, we won’t shut up about one thing - iterate. It’s important to understand that you can’t create a great product if you don’t keep optimizing it. The market changes and customers do too - they’ll want different things every year: just think of how COVID-19 impacted our lives and rejigged our priorities. That’s why if you only evaluate your ideas once, optimization stagnates.
How to avoid this mistake:
To solve this problem, you really should start from the top, i.e. speak to your leadership first to make sure that they understand the value of iterative market research. You can then put processes and systems in place to keep coming back to your studies and re-evaluate them.
Pizza Hut is a great example of a company that constantly validates its ideas. Their marketing team used Upsiide to test marketing messages when the pandemic hit to ensure that they answer all customers’ questions and concerns, and they regularly pull from their list of potential innovations to iterate on them and test them. Read the Pizza Hut case study to learn more.
5. Choosing the wrong KPIs
This common mistake stems from the reasons we’ve raised at the beginning - lack of strategy and lack of focus. Both of these factors suggest that you don’t have certain goals in place to guide your thinking. This results in an inaccurate choice of key performance indicators (KPIs) and metrics to gauge the success of your concept testing against.
KPIs are important numerical scores that tell you if your idea performed well or not. They help marketers and researchers to determine the success of their concepts based on parameters like appeal, usability, purchase intent, etc.
But when you have unclear goals, it’s hard to define which KPIs you want to look at in the end. How does your research tie back to your business goals and metrics? How do you know what success looks like? Not choosing the right KPIs will make your analysis more difficult.
How to avoid this mistake:
Work with top executives and managers from the start. Create SMART goals and make sure that you align your KPIs to those goals. Outlining your goals and KPIs in a dedicated brief is a great starting point to writing a good concept validation survey. Describe what you want to achieve, why you’re running it, and how you will define success. You can fall back on this brief when coming up with survey questions and considering how you’ll present the concepts. Peer review your brief along the way to make sure that your study has the right focus.
6. Choosing the wrong methodology
Finding the right way to test ideas is another decision marketers and researchers must make. If your methodology is wrong, your respondents might not understand their task or simply have a bad survey experience.
For instance, if you’re validating your concepts to know how appealing customers would find them, it’s best to do an idea screen, which will show your concepts one by one and allow respondents to compare and choose. However, if you’re more concerned with understanding what respondents think of each of your ideas, a simple idea screen might not suffice. In that case, monadic research is a better option because it lets people answer follow-up questions about each idea. And if you have extra budget, qualitative analysis and focus group interviews will give you even more verbatim data to work with.
How to avoid this mistake:
Using the brief you’ve developed as a guide, find out which methodology works best. It’s a good practice to learn more about different types of research or lean on your consumer insights team for expert advice.
For example, Upsiide is a great tool for screening ideas and assessing their potential against KPIs/measures of success. We’ve actually built out an entire suite of concept testing templates. But if you want to get some more detailed qualitative data, our colleagues at Dig Insights, the market research consultancy that created Upsiide, can help.
Validating product ideas the right way
The common mistakes in concept validation we’ve outlined often stem from a companies’ unwillingness to adopt an innovation-driven culture. When the company is aligned on making innovation a core part of their values and internal process, they are less likely to face some of the common pitfalls associated with validating ideas.
Want to learn more about Upsiide’s concept testing templates? Book a demo with our team.
