So you understand Lean Startup and customer development. You see the value in avoiding wasted time and money. And maybe you have even started conducting customer development interviews.
But how much data do you need? How do you know when you’ve validated or invalidated something?
I get asked how many customer development interviews is enough all the time. Here are a few important considerations:
Customer development is both an art and a science
The short answer is that there is no answer. (Don’t you just hate it when that happens?!)
Evaluating the results of customer development and determining how to proceed requires both looking at quantitative data and making decisions based on qualitative factors and personal judgements.
Sometimes customer development will not give you a definitive answer. You can almost never know for sure which decision is going to lead to the best outcomes, but you can increase your chances.
Lean Startup is a methodology you can use to guide you. There is no textbook step-by-step process to follow that’s applicable to all businesses all the time.
Look for intensity
Somebody pounding their hand on the table telling you how much they hate doing a given task or how much they hate a given solution they’re using to solve that problem is much more insightful than someone checking the “yes” radial button on an online survey.
Even 50 customer development interviews is not statistically significant. Use your judgement looking not just at the quantitative data, but at the qualitative feedback.
Look for variance
It’s important to look not just at the number of interviews that validated a given hypothesis, but to notice how the number compares to other hypotheses.
For example, if you conduct 20 customer development interviews and 15 people complain about problem A, while only 2 people complain about problem B, you can be more confident that problem A is the bigger problem. It’s not that 15 is a magical number, it’s just significantly greater relative to problem B.
Customer development is not a silver bullet
Everybody wants an exact step-by-step process. They want an exact number of customer development interviews to conduct or validated experiments. I wish I could give one.
The reality is, customer development will never guarantee your success. Lean Startup is simply a process to help guide you, help you use your time and money more efficiently, and ultimately increase your chances of starting a successful business.
Don’t let the lack of definitive answer be an excuse for not putting in the effort.
More is (almost) always better
I have only ever met one company that I felt was doing too much customer development. They had solid validation of their solution, but they were not executing on bringing it to the market and building a business. It seemed they were doing this out of fear. They didn’t want to face the music that customers might not be willing to actively use and/or pay for their product. Basically, they kept doing customer development for too long even though they had enough data to know what to do with it.
The other 99% of people I’ve met are not doing enough customer development.
According to Eric Reis, “If you have to ask whether you have Product/Market Fit, the answer is simple: you don’t.” Asking if you have product market fit is like telling people you’re cool: if you have to do it, it’s probably not true.
So, use your judgement, and when in doubt, do more customer development!
