Why do so many great innovations fail?
...the product failures had nothing to do with the innovations themselves. The cause of the low adoption rates by consumers had to do with the psychology of making decisions. The failed innovations were often superior to what was currently on the market and offered a distinct advantage to the consumer. Despite this, consumers resisted adopting the new product, causing it and, ultimately, the company to fail.
Imagine: you’re about to launch a new product. Your idea is great. Everyone agrees. It’s on-trend, relevant, and in a growing sector of the market. And it’s got an exciting USP, something that no one else in the market has done before. Investors backed you; collaborators joined you; established companies partnered with you. Momentum is building. Launch is about six months away. Everything’s ready to go.
But as you begin moving into production, you start feeling uneasy. You look at your design again: how, exactly, do you know that your target audience will like it? And thinking back, your original idea was great, but is that really a guarantee that your customer will think it’s great, too? You spoke to friends, colleagues, industry insiders, entrepreneurs, marketers, designers - but the actual target audience’s input was minimal. Think of all that money you’ve invested. Think of your backers. Think of your family, your friends, all convinced that your new product will be a hit. Think of your customers. It’s 2021 - shouldn’t all of these hopes depend on more than just...a hunch?
Markets move fast: the average lifespan of an S&P 500 company is just 20 years
Behaviour change is hard: American families habitually buy the same 150 items, about 75% of new products fail to make $7.5 million in their first year.
Most new products fail: According to data from Neilsen, around 85% of new consumer packaged goods fail.
This is the problem facing countless entrepreneurs, start-ups, and innovators every single year. Despite the growth of technology, a lot of innovation is still reliant on educated guesswork, gut feelings, or a limited form of testing. And so the failure rate of innovation remains high. But why is innovation still plagued by failure, and what can you do to make success more likely?
Does business size matter?
Innovation is hard. Even established companies with household-names, expert marketers and millions of dollars in advertising regularly fail when it comes to launching new concepts. From C2 (a short-lived predecessor to Coke Zero) to Amazon’s Fire phone, the history of marketing is teeming with examples of well-backed but poorly-received innovations. So being a large company isn’t enough...but neither is being small and ‘nimble’.
There’s a tendency to believe that small, nimble start-ups have a much better chance of innovating successfully, because the most memorable innovators are often plucky newcomers. The rise of Silicon Valley led to a veneration of agile ‘disruptors’ transforming stagnant industries with new ideas. Yet for every unicorn there are a thousand mules - it’s only because most start-ups are so small, we don’t notice when the vast majority of them fail. Which leads us to overestimate the power of start-ups, and minimise the risks. Ultimately, size is less relevant than most people think.
Innovate better
So if size isn’t the answer, why do so many innovations fail? In a landmark report, consumer insights agency Nielsen identified three common causes of innovation failure:
Neglecting to address a broad consumer need
Failing to provide a good product experience
Providing insufficient marketing support
All three can be prevented by engaging with your target audience - especially through rigorous in-market testing. A/B testing with successful competitors can reveal the scope of the need you’re addressing; your product experience can be tested through a qualitative survey; and marketing support is more successful when it’s informed by consumer insights, which are best gained through interaction with your target audience. Iterative testing isn’t just a way of honing products or concepts. By revealing the preferences, preconceptions, and predilections of your audience, iterative testing can power an entire brand methodology, reorienting your business so that it’s customer-centric, through and through.
That doesn’t mean you shouldn’t innovate. But it does mean that you need to be informed. In this report, we’re bringing together the best insights from retail, marketing, digital, and sales, to explore the most important questions around new product development. Why do so many innovations fail? What does innovation look like in 2021? And how can you really be sure that your target consumer will adopt your new product or service?
Here’s what innovation in 2021 will look like…
McKinsey & Company:
It is critical for companies to overinvest in rediscovering what matters to customers now
The Economist:
Innovation is crucial to productivity, [but it] is concentrated in too few firms.
Harvard Business Review:
...although most companies have come to understand the importance of looking outside for innovation, they have serious misgivings about how to do it.
Tinuiti:
Harnessing your data and using it to power personalized campaigns will be imperative in the coming years. Unfortunately, capturing that data is still a massive challenge for many brands.
Industry Week:
...companies will seek to develop new process innovations and faster, less expensive methods of designing products and getting them to market.
National Retail Federation:
...using on-demand strategies to create products lets brands respond faster to changing customer demand...the challenge is good data and technology that allows companies to optimize that information.
Deloitte:
But the scale of change is startling: the first half of 2020 saw five years of growth in online market share in a matter of weeks.
...Personalised campaigns based on live customer data, fuelling innovations that are tailored to behaviours that are changing faster than ever before.
So which trends are reshaping retail?
#1. Social commerce
E-commerce has been around for over two decades. But social commerce is taking it to the next level, allowing brands to create storefronts within social media platforms like Instagram in much the same way they would create a shop on Amazon. Not only does it allow consumers to shop seamlessly without leaving their social media streams, but it also enables brands to create more intimate and personalized retail experiences.
#2. On-demand data
Customer data is undergoing a transformation. While focus groups, surveys, and customer testing are still popular, companies are supercharging their learnings through platforms that give them real-life access to customers, making testing instantaneous, speeding up product iterations, and allowing products and services to be more tailored to their users than ever before.
#3. Direct to Consumer (DTC)
The pandemic has accelerated the shift to digital. Companies whose business model avoided bricks-and-mortar retail, preferring to sell directly to customers, have been boosted by the impact of Covid-19. With contact-free services in greater demand than ever, these businesses find themselves better able to cater to their customers - and with a direct connection to their users, they can gather better data from these customers to better inform future innovations.
#4. AI and machine learning
Machine learning isn't always obvious - it's gradually reshaping retail from within. More and more companies are using algorithms to anticipate demand, predict consumer preferences, and improve their service. From chatbots to augmented reality, AI is enabling smart businesses to gain deeper insights into customer, market, and industry.
Why these trends are all related
Industry Week:
[prediction for 2021] Rather than waiting until the end of the process, companies will enable users to share “in-process” designs. They will also gather customer feedback and alignment earlier to allow for more iterations and ultimately a smoother approval process and increase their chances of getting their product correctly aligned for the market.
Innovations don’t succeed just because they’re great ideas. The most important heuristic for success is the extent to which they meet the needs and demands of their target audience. And to meet these demands, you have to understand your users on a deep level.
Ultimately, each of the trends earmarked as being trendsetters in 2021 - DTC, social commerce, machine learning, on-demand data - all share one vital similarity. They either depend upon consumer insights, or they expand these insights.
So while innovations like social commerce or machine learning might feel like groundbreaking new forces in retail, beneath the surface they’re engaged in the traditional endeavour of marketing: to understand the consumer, discover their true needs, and then to meet those needs in an intuitive way.
When new products fail, it’s rarely the fault of the innovations themselves. Most new products are superior to what’s already on the market in one way or another - that’s why their founders launched them in the first place. Low adoption rates are more often the result of inbuilt biases and behaviours within the consumers. These psychological factors create a resistance to new products and the change in behaviour that they demand. So innovators have to overcome these obstacles - but you can’t overcome what you don’t perceive.
Unless you really understand what motivates your customer, unless you realise what’s going on in their heads as they browse your products, unless you know why they prefer one proposition to another, the power of innovation is severely limited. Direct to consumer models aren’t much use if you’re offering the wrong thing to those consumers. Machine learning is misleading if you’re asking the wrong questions. Social commerce is risky if you don’t understand how consumers behave on social media. And gathering on-demand data requires being open to consumer insights in the first place, and allowing them to continuously shape your design process.
Great ideas aren’t enough
Innovation is rarely the result of one company working in isolation. Entrepreneurs might like to believe in the myth of lone genius, but breakthroughs are much more often the result of painstaking, behind-the-scenes work by a number of organisations, often in competition with one another. The question isn’t so much why innovation succeeds in and of itself, but why certain innovators emerge victorious from a crowded field of competitors. Why Uber and not Lyft? Why Google and not Yahoo? Each case is unique - but the most common underlying similarity is how well these innovators aligned their new developments to the actual needs of their users.
Even being first doesn’t guarantee success. RIM were developing smartphones over 7 years before Apple released the iPhone, holding 20% of the global smartphone market by 2008. By 2016, their market share had plummeted to 0.1%. RIM lost out to Apple for many reasons - but most of those factors can be summarised as ‘Apple was better at meeting customer needs’.
Ultimately, many great ideas fail to achieve their potential. The history of innovation is filled with forgotten products and services that genuinely offered advantages to users. Without understanding the actual psychology and motivations of those users, even advantageous products fail. Dean Kamen, the founder of the much-lampooned mobility scooter Segway, claimed that ‘Segway will be to the car what the car was to the horse and buggy’. But their comical, unnatural appearance meant that most people felt too self-conscious to use them, despite their speed and durability. A number of high-profile accidents, the poor state of sidewalks and pavements, and the high price tag did the rest.
Nothing can guarantee that your innovation will be a success. Yet a rigorous, multifaceted understanding of your target audience gives you the best chance. More than having the best technology or the most impressive team, possessing genuine consumer insights - of their tastes, habits, ideas, and prejudices - makes an innovation much more likely to catch on. It’s not just about changing the colour of your packaging, or choosing the name that your target audience seems to like most. Consumer insights can reveal hidden information about, say, how customers would prefer to pay for the service, or what sorts of propositions would make an otherwise-desirable innovation untenable. These may not be obvious or even rational - but they have the decisive power to influence whether or not your audience adopts your innovation.
Testing: the bridge between insight and innovation?
Nielsen:
The difference between a strong and a weak product experience can result in sales difference of 30% in year one. Closing this gap through fast, efficient pre-market testing has considerable advantages over experimenting in-market.
Yet if most people underestimate the power of testing, it’s often because they have an outmoded view of testing as a discipline. In the past, testing meant months and months of tedium: weeks of laboriously organising focus groups, surveys, on sample sizes that were sometimes too small to trust. In some cases it would take years before an insight was developed. Nowadays, most innovators simply don’t have the time to test in the traditional way: the world moves too fast.
Now, testing is easier, faster, and cheaper than ever before. Representative quantitative research is available digitally, giving innovators immediate access to a huge pool of consumers from around the world. And tests are more customisable than ever before, making the results richer. Companies can receive much more relevant insights by tailoring their testing. Done skilfully, testing enables companies to get the richness of qualitative research without giving up the precision of quantitative.
Taste testing Everipe
Smoothies have been growing more and more popular, and it’s easy to see why. They’re tasty, energising, and good for you. But as anyone who’s had to clean out a smoothie maker will tell you, they make a lot of mess. Everipe found a way to make them easier: freeze-drying nutritious smoothies, and sending them directly to consumers. The founders, Greg and Kerry, counted advantage after advantage: freeze drying maintained taste and nutritious content, and it also created far less waste. You could even store the freeze-dried smoothies in your pantry up to a year without spoilage - they didn’t even need to be refrigerated. The benefits seemed irresistible.
‘We'd always heard that our concept was exciting’ said Kerry, ‘but we struggled to prove it’. The proposition was enticing; but it didn’t translate into a product portfolio, brand, and marketing strategy relevant to consumers and investors alike. With a background in consumer packaged goods (CPG), Kerry knew that testing could bridge the gap between innovation and insight. By rigorously testing competing propositions and adapting them accordingly, Everipe could sharpen their proposition and elaborate it into a convincing brand line-up. They could develop marketing strategies that they knew were relevant. They could communicate their product’s natural advantages in a way that was relevant to consumers. And they could use the consumer insights to get investors on board, demonstrating the long-term potential of the brand in the minds of the people who matter most: their target audience.
Sip by sip
Everipe didn’t want abstract testing. They wanted actionable in-market insights. They wanted to measure the intuitive, split-second interest of consumers, just as their products would be treated on-shelf. They wanted to understand just how appetizing consumers found their proposition, as compared to direct competitors. And Everipe wanted to begin understanding their audience in greater detail, identifying the different sub-groups of consumers within their target audience, enabling them to shape new product line extensions to specific types of smoothie drinkers.
They used Upsiide, Dig Insights’ innovation testing platform, to screen their ideas and test their concepts in every format, from their packaging to their messaging claims and logos. Upsiide’s user-friendly swiping methodology enabled Everipe to gauge the intuitive attractiveness of each smoothie flavour. In turn, this allowed them to quantify the in-market appeal, beefing up their pitch to investors and revitalizing their own confidence in the proposition.
It was a pivotal moment for us to hear from a robust sample set. [The data shows that] we have a very powerful idea. So we're now spending the dollars to bring a team of consultants in, solidifying our brand and products, and scaling production. I'm not sure we would have had the confidence to invest without the clarity that Dig's study brought us.
Testing doesn’t have to be a slow, momentum-sapping death-by-focus-group. It can be both fast and rigorous, with quantitative integrity and qualitative generosity. It can sharpen your brand proposition, shorten your time to market and supercharge your investment efforts. But to unleash these benefits, it needs to be iterative: in real-time, with in-market insights, and capable of being refined over and over again. That way, you can get the fine-grained test results you need to develop products and propositions that you know your target audience will actually love - pleasing founders, investors, and consumers alike.
The future of product innovation? It’s relationships
More accurate and voluminous data about shopping patterns are breaking down the decades-long relationship between mass consumption and mass production...producers and consumers will have a more direct relationship with each other.
On the surface, the vast majority of retail looks like a transaction. You choose a product, exchange money for it, and then use it however you want. Loyalty schemes and customer services establish more of a relationship with consumers, but generally they’re pretty weak. Marketers sometimes like to think that they have a relationship with consumers, but it’s rarely reciprocal.
The latest wave of retail innovations will turn more and more transactions into relationships. And rather than starting the relationship with consumers at marketing or point of sale, new technologies and approaches are allowing it to start far earlier - and to become far richer, deeper, and more personal. Brand loyalty is what marketers want most. They already know that, to develop brand loyalty, customers must feel seen, understood, and served with products that meet their needs. But how can you meet the diverse needs of different individuals, even within a single target audience? This age-old problem has been addressed in part by data, AI, and the new models emerging from them.
Take direct-to-consumer (DTC) models. They’ve become one of the pandemic success stories - not just because customers have been kept indoors by stay at home orders, but also because their model allows them to gather data on customers as they shop. In turn, this enables businesses to iterate on products based on the insights gained from the intimacy of their service.
Unlike traditional retail models, a brand selling energy bars direct to their consumers can track each user’s consumption, preferences, and habits, enabling them to build a much more personal relationship. Not only does this increase engagement and sales, but it can also enable consumers to shape how brands design, market, and sell their products. In a bid to create that powerful one-to-one relationship with its customers, in 2017 Nike began withdrawing from its wholesalers. Now, DtC makes up 40% of Nike’s sales. Consumers now expect a more personal approach from brands. Firms lacking this personal touch are likely to be overlooked in favour of ones that know their customers intimately, and can communicate this connection meaningfully.
It all starts with understanding your consumer - and that starts with testing. In fact, from social commerce to data on-demand and AI, every major innovation trend of 2021 will boost the power and significance of testing. These trends enable brands to test more accurately and iteratively, giving them instant feedback on propositions, which in turn can yield tangible advantages.
Five tips for innovation success
#1. Test everything - not just sales and marketing
It’s clear. For innovation to succeed, it must be informed by real market insights. Yet if those insights are gathered right at the end of the innovation process (when deciding how to sell and market your idea) their benefits will be limited. By that point, much of the most important questions will already have been decided - without properly researched input from your target audience. Your preconceptions will also shape the types of questions you ask during your testing, meaning that you could miss out on more important avenues of enquiry entirely. Ideally, your customers should be influencing the entire innovation process. For example, in China new product development is being led by a ‘consumer-to-manufacture’ (C2M) model. Here, data gathered from e-commerce platforms is analysed by AI to identify potential shopping trends, with the resulting insights driving rapid manufacturing, shaped by those consumer insights. Algorithms can identify previously-obscure gaps in the market, unaddressed customer needs, and hybrid products with good chances of success. These can then be tested among focus groups, sharpened according to more direct insights, and sold to consumers within a matter of weeks. It all stems from direct, proven, and data-driven customer insights, improving margins and reducing the rate of failure. While C2M isn’t for everyone, it does highlight how harnessing customer data early on in the innovation process can hone your idea so that it’s much more relevant to customers - and much more likely to succeed.
#2. Whether qualitative or quantitative, it must be quality
An innovative idea is only as enticing as the human insight on which it’s based. And that human insight is only as reliable as the data behind it. Selecting the right testing methods for the job, therefore, is vital. Yet too many organisations are biased towards either quantitative or qualitative methods, when both play an important part. Generally, for testing a particular hypothesis, quantitative methods give you the necessary sample size to identify broad patterns of agreement or disagreement. Whereas for exploring open-ended propositions, qualitative data gives the richness and detail you need to uncover the hidden perceptions that can shape the intuitive likes and dislikes behind shopping. Measuring opinion or in-depth investigation of the motivations behind opinion: both can help to ensure that your innovation succeeds. And both must be shaped by properly-considered questions and testing parameters. Asking the wrong questions or using improper methodologies will give you the same volume of data, but it won’t be as trustworthy or meaningful as it should. The interface matters, too - the way you approach consumers makes a difference to the answers you’ll receive. Choose an interface that’s natural, intuitive, and recognizable to your audience, and you’ll gather better data.
#3. Sharpen your questions, shorten your lead times
Contrary to the Silicon Valley narrative, innovation tends to fail when it’s rushed to market. In-market iteration can help to iron out initial flaws, but outside of software, post-launch product tweaks tend to be costly and time-consuming. In a 2018 report, Nielsen showed that products with a strong initial performance with users enjoyed 30% more sales. The lesson? Test, test, test - and conduct your tests so that they’re agile, relevant, and intuitive for consumers. By honing the questions you ask consumers, exploiting the power of AI to make sense of the data, and making use of tools that visualise the results in insightful ways, you can speed up the process. So that, rather than launching too soon, you can quickly harness consumer insights prior to launch, giving your product all the benefits of iterative evolution without the growing pains.
#4. Dig deep into in-market insights
Too many organisations use data to back up what they already know. They’re convinced that consumers will love their product and so, just before launching, they do some quick tests to back it up. And, by using leading questions and closed assumptions, they get just the results they’re looking for. There’s no in-market research, no in-depth analysis, and no refining of the product. When they launch, sales are lukewarm. But without more testing, the founders won’t know where the gap between the idea and the poor sales actually is, much less how to bridge it. Rather than just searching for the results you want, it’s better to harness tools that can harvest the data for insights you might never even have considered. TURF analysis identifies potential portfolios and their architecture by generating hundreds of possible product mixes and their total projected reach. Interaction mapping can group competing ideas into distinct territories that are liked by different segments of consumers, helping you to evaluate which territories are most promising. Given that innovation has a much better chance of success when the concept has a broad appeal, understanding which of your propositions is capable of attracting the broadest customer base is vital.
#5. Be radically customer-centric
Innovation can feel like a casino where the cards are stacked against you. There are so many hurdles to overcome, so many potential pitfalls, that success can feel like blind chance. And with more channels, media, and variables to cater to, the necessary expertise seems to be continuously growing. Yet there have never been so many ways to reliably and accurately measure the one true guide to all of your actions: the consumer. Every new touchpoint is an opportunity to gain customer insight. Used wisely, every interaction with the consumer is an opportunity to test, iterate, and understand. But to do this, you have to trust your target audience. Think of your innovative concept as a first draft: the final version will be influenced and shaped by your consumer. So ask yourself: are you ready to trust your audience, to let them into your innovation process, and to collaborate with them en masse through testing? The answer could determine whether you’re a regular fixture in their shopping baskets, or just the flavour of the week.
A bit about us
For marketing and insight teams that need to predict which ideas, concepts, and designs will perform best in-market, Upsiide is the innovation insights platform that tells you which is the right move for your business’ future. Upsiide was developed by the insights pros at Dig Insights, who are passionate about reinventing the language of research, putting it into the context of today’s business goals.
Get in touch to see how we can supercharge your innovation process.
