Lean Startup Case Study: Idea to Product Market Fit in a Weekend

On the latest episode of The Building in Public Podcast I have Ajay Perumal. I met Ajay a couple of weeks ago at a Lean Startup Machine weekend seminar. And Ajay and his team made some incredible progress on validating their startup idea over the weekend. In this interview we’re going to dive into how Ajay and his team applied Lean Startup Principles and the customer development tactics they used to test, validate, and optimize their idea and ultimately reach some approximation of product market fit and even make money over the course of the weekend. Check out the highlights of the interview or listen to the interview here

Idea to Product Market Fit

Ajay: Before we get into the numbers, what we were able to accomplish is first of all, tweek the idea to something people actually want. I think the Lean Startup methodology was a great tool to do that including the Javelin Board.

We also got great customer validation. A lot of people said they like our product they wanted to buy it and this was as a result of all the tweeks at the workshop. We got a great vision of how we could proceed with this product faster over the weekend. And in terms of numbers, we did great as well. Website got over 5,000 unique visitors in less than a day. And we got over I think five hundred button clicks of people who wanted to sign up for a dollar. And I think was a result using this Lean startup  methodology. We condensed a lot of learning into a short period of time.

Mike: Amazing.  And where were you with the product before you went into the weekend?

Ajay: Well, nowhere actually. So the idea I went to the workshop with was essentially to build a sort of a selfie library. A library were you could share selfies instead of emojies. These ideas came more or less  a couple of weeks before the workshop while I was sitting on the couch my friend and I were taking selfies. I told him this is what I do these days instead of actual emoticons. He thought that  was pretty hilarious and we should use it as an idea and so, that’s pretty much where we were and I just decided  to pitch this Lean Startup Machine.

Mike: Awesome. So, the idea was solving a problem you had, building a product you wanted,  two weeks later going into Lean Startup Machine weekend, and over the course of  3 days the accomplishing all that.  Pretty amazing.

What was your familiarity with Lean Startup and customer development prior to the weekend?

Ajay: So, I’ve been a big fan of the Lean Startup methodology. A really good friend of mine works at the Lean Startup Machine so  he’s been going on about the benefits of the Lean Startup machine for a while. And I also casually attended Lean Startup machine in Newark. I didn’t actually participated I just went there to see what is it like, what the workshop is like so I definitely  wanted to  attend at some point and actually having an  idea to validate at  the  workshop gave me great opportunity to do it.

Forming Hypotheses

Mike: what’s the first steps you took in getting started?

Ajay: So, the first thing we do is to form a team and this was initially. So we had a team of of 5 people and our job on the first night was just to sort of design our experiments  for the next day and this involved identifying what pain points and problems our potential customers.

So, basically  set our customers as anyone who uses an emoji or a selfie. And we said that the problem is they thought that the current messaging and texting options were too limited.  And didn’t allow them to express themseves freely. And we sort of have to define measurement criteria for this because this was a little vague to begin. So anyone who has multiple texting apps.  We also set others assumptions that people were actually having more emotions than available through standard messaging format. So that was what we are trying to test. And then we go out and meet people. We set our success criteria: if 6 out of  10 people have this problem, then we have a product that could fit in the marketplace.

Customer Development Interviews

Mike: Okay, great and so, the next step, I assume Saturday morning was to go out on these customers, was it right?

Ajay: Absolutely! This was quite a challenge because Saturday morning was raining heavily and the workshop was held in Times Square. So we  went out in Times Square looking for people because this was we also have the generic customer base and we won’t have to necessarily have to  go to a particular spot to look for them.

What we very quickly found out was Times Square was not the ideal place to find new customers and do customer development interviews because people are selling stuff at Times Square all the time and this is  people won’t have to do with an interview. Nevertheless, we manage to speak to about 30 people, between the five of us and 21 of these people seem to have the problem as we define it. They use several texting options. But would love another app that would allow them to express themselves more  freely.

So we definitely wanted to persevere past this point but we also have several important learnings. The biggest one being that the word selfie was a taboo. People did not like admitting that they took selfies or shared them so we have to tweek that a little bit. People are a lot more comfortable using emojis and emoticons. So we said that would probably the direction we have to go with, if we are going to develop this product further. And we definitley realized that the demographic we have to target are the teens through college mostly because all the people did not associate with selfies or emoticons. One of the responses we even got was from this one guy we spoke he said, “I’m 30 and I don’t talk in pictures.”  So it was pretty clear from that so at that point we were able to aim a little lower in terms of age and demographic.

Finding Customers to Interview

Mike:  How did you go about actually gaining those interviews? You said that you did 20 or 30 like how did you decide who to go up and talk to and what did you even to stay around to be with the interview.

Ajay: This was a difficult exercise actually. One because of the rain, two because this is Times Square. So initially we said since we didn’t know any age or sex demographic we just said we just go out and anyone and everyone. And then try to interview them and then based on a learning narrow our demographic down. So I think we tried several approaches we said, “Can we talk to you for a few minutes?” That didn’t work at all. We said we’re developing product that didn’t work too well at all. We  found that the easiest way to get people to answer you is you just say you’re a student working on a project. Because we found out that people would just like to help students little more.

We told them that we’re working on a project  we love to interveiw them, people are pretty skeptical at this point. But since it was in the rain and they were actually waiting in lines they really speak to us for a little while. At this point we just generally try to, try to be a little bit more friendly. Sort, of ease their tensions a little bit because we realize they’re just too scared to talk to us as we were to talk to them. So, we just started to ask them what they’re doing, who they are, as a person, and then asked them our customer development questions. Pretty much just asked them questions pertaining to the product which was what their messaging habits were whether they use selfies.  How many messaging apps they use. We also prepared a sort of template just to guide us on which track we should take when asking any questions. And then at the end of the day, we just pitched them the product itself, but we didn’t really use just to validate our learnings but we said if you have this will you use this? Jut to gauge what people were thinking. But we try to avoid using that in developing our learnings from these experiments.

Customer Segmentation

Mike: You mentioned that your initial customer segmentation was very broad…so you talked to a variety of customers. And then you narrowed down on younger teens. Can you talk more about what made you decide to narrow down your customers segments? How were you able to get to that customer segment?

Ajay: Absolutely. So everytime we interviewed someone, we started to take note and said what their demographic was, and who were these people answering to our questions. So, anyone who was our customer or a texter who uses emojis or selfies. And first of all, what we found that anyone who even   fell into that broad classification we use was a pretty much a younger generation and who were teens who were initial college goers. And what we found was among these teens the older ones were the ones that didn’t seem to have the problem we had.

So, anyone we spoke to pretty much was about the 30 year age gap, either were reluctant to admit that they would want to use several emojis or selfies. Or just pretty much dismissed the idea that they don’t want to do so in the future.

Mike: Perfect! So it was interesting how by doing customers that you were able to discover your customers. So not only were you testing your idea and gaining insights, you’re figuring out who your customers are, to begin with.

Ajay: Right. And this changed so much from the initial idea we had even when we ran into this experiment we just put down different customers, we thought would fit this target and our assumptions varied from all people who texted their grandchildren to the teens and college goers. We had no solid way to narrow down on these customer segments. Until we went into these interviews. And these interviews pointed us in a very, very specific directions.

Mike: Amazing! Identifying your customers can really help you focus your marketing in the future. If you’re going to go out and market and get the product to all these different groups you know, you might stretch yourselves thin and realize you were wasting money to market to the customer’s that don’t want it. But because you do customer development in a weekend, now you can be a lot more efficient and effective.

Customer Development Questions

Mike: Can you talk about some of the specific customer development questions that you ask during these interviews on Saturdays?

Ajay: Our initial questions revolved around just understanding the persona…so we asked them their names, whether they lived in New York. Visiting or whether they actually lived in the city, this was just pretty much to get them comfortable because they were skeptical when answering the questions in the first place, and we thought this was really important. Then the first question was to qualify them as a customer…whether they’re a texter and how often they used text. Then the next question revolved around why they texted. If it’s a form of recreational communication, It was the target customer we we’re looking for.

Then once we have established what texting habits they have, we asked them whether they use emojis or selfies in communications. And again, this is when we found out that people  didn’t like to admit they used selfies. We found that most people do admit it once we got to know them a little better.  But they were a little hesitant earlier on. And then we sort of have these 3 criteria which would define whether the customer had our specific problem. As we define it. And they were whether the customer had several texting options. Whether thay have Snapchat or WhatsApp. These should mean that they have that problem. And they used several emojis or they used the combinations of emojis. And another one we asked is whether they have conversations and emojis and surprisingly all of these people who fell into our customer back-end said yes. So this was a definite indication that these people filed into our customer segment and had this problem.

Mike: Okay. Great. So you have one or two rapport building questions which I think is a good strategy. Then you did your customer segmentation questions, so you make sure you understood who you’re talking to and whether or not they fit into your target demo. And then you did some problem validation questions.

Recapping Customer Insights

Mike: After these initial round of interviews, what happens next when you regroup the team? what did you learn and what did you do next?

Ajay: Right, so we got back together and we try to figure out what each of us learned, because we split up into two groups, so we started to exchange notes to see what we thought and learned. The good thing was between the two groups, we have consistent learnings. So, that definitely helped us a little bit. Smoothen the process a little bit. The one learning that we had as we mention earlier was, the demographic was sort of a younger age group teens or college goers. All these people tend to use emojis an awful lot. They used a lot of combination emojis. And the third one was the App or tested selfies.

So we sort of had this good direction to go with moving forward we had a definite customer at least that we could nail down as teens or college girls. Teens or  college girls that used emojis specifically. And this help us to find problem from next set as well. And this time we decided to focus more on the emojis and emoticons. And avoid the whole selfie part altogether. This was a drastic change from how I envisioned the idea initially. And we were able to do this within one morning.

Minimum Viable Product

Ajay: The next problem we defined was that emoji’s are unoriginal, too mainstream and were not personal enough for people to use. And we try to define a solution and the solution was that you have to have a custom faced keyboard. So you have a keyboard with your face on it. And these nominal emojis would be replaced by your face so you could share this and have fun with it. And the assumption that we have here is that people would actually want this. We did know from the early experiment that people were open to new forms of communcations. But our assumption here was people would actually use a human like emoji.

We set our success criteria to if 6 out of 10 people signed up give us their e-mail address, then this would be the product that people would want. But then, we very quickly realized that just by describing an idea that wouldn’t really cut it.  So, what we did was created a mock-up on PhotoShop and we just put our faces on there on the keyboard and replaced the original emojis and decided to just go out on the street and show these to people. So we had this, PhotoShop image on our phones which really look like a screenshot so people thought that this app actually existed and we just tried to show these to people on the street and said would you use an app like this? So what we did is we put our face, our team members’ faces and a bunch of celebrities’ faces on this keyboard. Just to see if people will be going to say yes and they will use it. And then we pretty much went out and tried to conduct this experiment.

Mike: Awesome! So it’s pretty amazing you were able to hone in into the problem. And the problem being taking selfies and emoticons not being personalize enough or having enough flavor. I love this experiment you’re running — how you were doing mock-ups where people think the product is real.

How did that experiment go? What kind of reactions were you getting? You had a success criteria of 6 out o f 10. How did these experiments stack up?

Ajay: Right, so these experiments did actually really, really well. So, I went out alone on this one, and then met up with one of my other team members half-way through and this one just involved asking them the same questions to begin with as the first experiment that make the rapport building questions and then just show them the screen shot and see what they would react people loved it they said, we have reactions like this is so cool. This is awesome, I would definitely use it. I would want my friends face on that keyboard, where do I get it this from. One of the people even said can I work for you. So it was really a positive response I think we got 33 out of 52 people give out their e-mail address but we also have a very interesting learnings.

People really didn’t care about the celebrities on the keyboard they were more excited having their face there. And the ability to put that friend’s faces on that keyboard and share that. And the celebrities didn’t really strike a chord on these people. It was more a social and customizable feature of this keyboard that really appeal to them. And that was pretty much what we learn from this experiment.

Mike: That’s amazing!  They were excited to use this product. They are asking you to see the product, which is strong validation and on top of that you gained some valuable insights which was that people don’t care about the celebrities that much, and they want their own faces. That really helps you shape your product.

Okay so you guys your strong validation here you met your success criteria what happen next?

Ajay: The interesting thing while we were conducting this experiment while walking around  the street trying to find customers to show these and all the way we bumped Trevor who is the founder of Lean Startup Machine, and we told him pretty much about what we’re doing and we were very excited we were having sign-ups we showed him the app and then he busts our bubble. He told us that we were leading people on in showing them an app and we were saying and asking them questions, such as would you use this? And asking their e-mail sign-up and he said although we’re getting some validations this is probably not the best validation that we are getting. So he suggested that we go onto a landing page. And our whole next experiment was based on his comments.  So, again we got back to the workshop and we started to exchanged ideas and we all pretty much again had the same findings.

They like the fact that they could customize their keyboards and they can change that wherever they want it. So the next step will be building up a landing page. We kept the customers we said and the problem was emojis are unoriginal and the solution was the custom keyboard.

The One Dollar Test

Now the question was are people willing to pay $1 for this app. So we went and built a landing page. We make sure we put the screen shot there just so people can get the vision of the product they were buying and we definitely put the pain points of this customer on this website. So we put down those little box on the bottom tombstones just listing down those the keywords that we found and which is just and has to be customizable and also has to be sharable. And we tried to emphasize as much as possible to get people actually by what was appealing to them.

Mike: Excellent. Based on learning you were able to build a landing page that you know is likely to convert much higher. You now have a product that is more in demand by customers because of these insights that you gained. So asking people to pay one dollar I think this is super valuable. Not necessarily for revenue but so you can truly validate. I like e-mail addresses which is a small form of validation but now asking for a dollar, just one dollar, it makes a big difference. When you asked people to put their money where their mouth is you start to learn from this and you really can get some validation. So how did these experiments go and what did you do exactly?

Ajay: This experiment was really interesting. Once we put up the landing page with three key points at the bottom saying sign up for a dollar. And then we sort of blasted out on Twitter. We try to conduct a mini experiment within this experiment, and so one of the team members who work for the gaming industry… so what we did was to create another Photo Shop mock-up with all the Call of Duty characters on the keyboard and players on the keyboard and he blasted out on Twitter and it just blew up. These guys went crazy and they absolutely loved it.  After going crazy on Twitter, we hit I think  2,000 unique visitors on the site. And 103 people clicked the sign-up button. And I think in less than a couple of hours of us sending this out on Twitter.

Mike: Wow! So, you tested a new customer segment and in this case gamers. Even more specifically, Call of Duty gamers. And you sort of blew up on Twitter.

Ajay:  The learning we got here was it doesn’t matter what’s on the keyboard. It just has to be more personalized.

Mike: I understand you were able to get some paying customers in person. How did that go down?

Ajay: So that was pretty really interesting. So, while this is going on, I think one of the mentors working at the site suggest why don’t you go out and get people to  pay a dollar. We thought this was a little absurd but to start with, but then we said that we have the time so why not. We made this flyers and we said if anyone sign up for the app now  and pay the dollar they would receive benefits, they would be the first one to get the app they won’t have to pay for the app when they came out. And one of our team members just went out to the street. So I think in about ten minute or so go $7 which is great validation. He went out the next day and got some more money. I’m not sure how much, but he was getting money almost immediately. He just showed them the flyer and people are more than willing to give him money. This one person even gave him a dime because that’s all she had so he took it anyway, but it just went to show how strong the validation was.

Mike: Yes, it’s amazing because  people are willing to pay and it sounds like you know people were excited to pay. You know you’re really working on something that people want. That gives you the confidence in your idea and it gives you an indication that you should be spending some more time and money on the product idea. You know you’re not wasting time and money on something that nobody wants.

Do you plan to move forward with it?

Ajay: Oh, absolutely. Between my day job it’s a little bit difficult but me a couple of friends I’ve been working on it and I’ve been in touch with a few I grew up in India I have a few friends who start developing companies back home so I’ve been talking a little bit trying to see the feasibility of  getting this develop so even going forward we want to be as lean as possible because I think that’s one huge take away we’ve taken from the workshop. So we want to build some kind of proto type as simple as possible as low cost as possible and maybe send that proto type out to all the people who signed up from the website. And then see what they are thinking then through that feedback sort of treat that idea a little bit then see where we go.

Listen to the full interview on iTunes.