Nabeel is a Venture Partner at Spark Capital. Prior to joining Spark, Nabeel spent 15 years as a startup founder, CEO and head of product at the confluence of design and technology.

TOPIC NABEEL HYATT COVERS

→ Has a reputation for understanding growth, speaks, teaches, and writes about it

→ Recently wrote an article suggesting renaming growth hacking to evidence-based product development

→ Discusses the importance of using metrics and data to drive growth in consumer

→ Emphasizes the importance of discipline and taking the time to reflect on the results

→ Suggests using evidence-based product development as an effective approach to growth

→ Discusses the use of metrics and data to drive growth in companies

→ Mentions the use of A/B testing and controlled samples

→ He advises focusing on understanding customers and using data to inform creative decisions

→ And a whole lot more

LINKS & RESOURCES

→ Nabeel’s Website

→ Spark Capital

WATCH THE INTERVIEW

READ THE TRANSCRIPTION

Bronson: Welcome to another episode of Growth Hacker TV. I’m Bronson Taylor and today I have the Bill Hiatt with us. Nabeel, thanks for coming on the program.

Nabeel: No problem. I’m excited to be here.

Bronson: Yeah, we’re excited to have you. You’ve done some cool things. And for people that don’t know. I’ll rattle off a few of them here. Currently, you’re a venture partner at Spark Capital, and previously you were a general manager at Zynga. And you ended up at Zynga through an acquisition when they acquired your company, Conduit Labs. Does that sound right?

Nabeel: That that sounds exactly right. Yeah. We built a gaming company early on, sold to Zynga, and then I was at Zynga running a couple of games up to the IPO and afterwards.

Bronson: Yeah. And then before that, I mean, if we went back even further, you got even more interesting things do it MIT Labs. I think there a spinoff of ambient technologies or something like that. You got some mighty awards for being a great innovator. So there’s, you know, there’s the whole resume there if we want to keep going back. But your recent history is enough to give people a taste of what you’ve done. Now you’re one of the VCs that’s kind of gained a reputation for understanding growth, you know, different VCs bringing different things to the table. And because of your history, you just kind of have a knack for it. It seems you speak on the topic, you teach on the topic, you write about the topic. So I want to start by kind of having you help us understand growth. It’s sort of a high level. So a few months ago, you wrote an article where you said we should stop calling it growth hacking and start calling it evidence based product development. Walk us through that. What do you mean by that?

Nabeel: Yeah, first of all, clearly I’m not like a marketing namer because I don’t think evidence based product development is going to catch on. It’s not something that, you know, rolls off the tongue. But there are two things that I wanted to really touch on that I feel like are really not encapsulated in the conversation that we have about, you know, growth. We talk about growth. We talk about growth hacking. One is, I think, best encapsulated by I think I might even mention the pose, a quote by David Ogilvy, which is like, you know, some people use statistics the way a drunk uses a lamppost, which is more for support and for illumination and a general sense that like like you’re you’re you really you kind of know what you want to do with your product anyway. And you just want to try and use numbers to either make yourself feel better or, or the kind of strange, retarded notion that, like, the numbers are going to tell you what to do, that you’re just going to stare at the number and that that’s going to make your product amazing and make people love your product and make you want to keep using your product. And and my worry is that because people get conflated with that definition, they lose what being a process or a metric mind can do for driving real growth, especially in consumer, but also in enterprise products. And so I think about this more as applying science to an area that was largely filled with superstition. You know, you know what you build next in your product. How is it going to grow? Is like five guys in a room on a whiteboard going, Oh no, this one or I think it’s this one or I think this one and and we want to fix that, right? We want to try and say it does come really amazing ideas do come just like a scientist. Imagine a scientist trying to figure out a cure for cancer. Right. The idea is still come from some deep creative place. That’s a mixture of them looking at other studies, them doing work on their own. But really they have a hypothesis. They say, you know, based on my knowledge, I think it could be this. And the hard part, the discipline part which startups have such a trouble doing is then going through the process of testing those hypotheses. So even though it comes from this creative place that this part at the end where you go, okay, so what do you think it’s actually going to do? Like I know you said, it’s probably going to grow. How much is it going to grow? Where in the product need to grow and grow by retention? Is it going to cause higher virality? What kind of morality and what channel being at discipline and then spending time after that future launches when you’re a kind of go go start but you keep doing, doing, doing. And I’m an engineer and company builder from a get it done kind of issue. But you’ve got to take the time. Just like a scientist would look back at that experiment and say it failed, it didn’t fail, and then be intellectually honest about why it did so that you can learn and get better in practice.

Bronson: Yeah, you know, there’s so many more companies now. They’re thinking about metrics more than a couple of years ago. Do you find that they don’t go and look after the hypothesis of what actually happened? Do you find that they install Mixpanel, they install KISSmetrics and then they kind of let it run and they never really have that moment of like, All right team, let’s look at what has happened and compare it to our hypothesis a month ago and see if it lines up. Do you see that they don’t do that a lot?

Nabeel: Well, I can say that, you know, as an entrepreneur, which is the bulk of my career, I found that last step the hardest to do. You know, I personally found it very hard to decide on a Friday to sit down for 2 hours with my team and go back through the not even the week, but the previous. We make sure we draw any results and then walk through the data. That is very, very hard to beat that amount of time and personnel up to doing something like that. But, you know, if you don’t, I think it is it is very, very hard to learn. You know, the nature of growth and even just future development is that most of the time, we most weeks we don’t knock it out of the park. Most weeks we release a feature and move some number up a little bit or down a little bit. That is really lost in the noise if you’re you know, if you’re not staring at it. And so, you know, everyone can learn from the, you know, every six months or a year we release a feature and it completely pisses off your users and you get emails and phone calls and your subscription rates go down. Like, it’s easier for a product person to learn there. Just got, you know, they just touched the stove and they got burned. They’ll learn and seem like, you know, you insert some amazing feature that suddenly causes growth. Like anyone can learn from that. Like, you don’t even look at the next panel display to see. But that’s that most weeks. And so if you’re only learning from those experiments, you’re only learning from those features. You’re only learning once or twice a year. And frankly, that’s just not you getting in touch with your customer fast enough. So what the reason you look at an AB test, you know, the reason you do a controlled sample where you only release a new feature to 5% of users. Then at the back end of that, watch what that 5% of those users are doing versus the control, which is your current user rate. The reason you do that is so that even if it causes some metric go up by five or 10%, you get a read on that and you understand hopeful and ideally your understanding at the end of the day a little bit more about your your customer, which is the goal.

Bronson: Yeah. No, that’s great insight about how to actually think about that. Now, as an entrepreneur yourself and as an V.C., do you find entrepreneurs have a hard time kind of balancing, as you put it, the soul and the numbers? Did they feel like an internal kind of friction there? Because some people, they want to be data driven. Some people they want to be create, you know, creatively driven. And yet we have to do both and somehow, somehow we have to marry the two. Do they find that difficult and how do they navigate that? What’s your advice to them?

Nabeel: Yeah, I mean, I think there are two people, two types of people that come to growth hacking. I think there are people who look at growth and and they kind of mostly want tricks. They like the type of person that’s trying to flip to the back for the answers in the back of the book. And and the problem with that approach, of course, is that any given trick is it’s a very short life, and whatever works now is probably not working nine months from now. And so you look for a sustainable process. And then when you look at a sustainable process, you’re talking about split testing and doing control and much other stuff. And then people worry that it’s getting in the way of the product. Is it is it, you know, am I still able to sell the product I really believe in? I think there are I have two suggestions for folks to try and guide their team in their culture when they’re trying to make sure that they build a product that both they love and their customers are going to love. The two ways that I think it goes off the rails are one that once you get results back from a task that you forget to insert your brain in the process. Right. That, that, that just because you happen to test a button being red or blue and it says blue, it’s not hard to say, okay, now we have to do blue. If you really didn’t like blue, first of all, we should be testing it. But but if you like blue, let’s have a conversation about why you think the customer picked that thing, why they went down that given path that might lead you to a whole new feature, which is actually it’s just your customers intent, right? Your customer is telling you something about the product that if you’re surprised by it, that wasn’t something you wanted to do it maybe because you just don’t. There’s some gap between what you believe your customer wants and what your customer actually wants, and that’s what you need to recognize it as not as a directive, not as a now to do X, but as guidance. And so that first the what I’m going to build in a given week, month, year, that should it can be performed by data, but it should come from your heart. It should come from some area where you really care and you really think it’s going to drive the result that you want, which is the second bit of that equation. So one is just understanding that that, you know, one is making sure you’re not going to the back of the book and trying to look for answers that you’re doing it because some other company yet, but because you, you know, you believe in it. And the second part of the equation that I think people sometimes they’ll lose their way is when they’re not being very clear with themselves or their team about where the bounding box of ideas is. You know, it isn’t enough to start a software company and say, hey, I think for gross growth we can sell ice cream online. It’s like, no, that wasn’t our vision and that wasn’t our mission. And I think people don’t take the time to be really material about what their vision, their mission is, what is outside of bounds, what is what are our values and what are the things that I don’t care? I don’t care what growth means. I don’t want to do online gaming and prostitution. Like, you know, I think I’m drawing very hyperbolic reference here, but I think you actually have to get fairly new. This is a type of culture we’re trying to build. This is trying to kind of company we’re trying to build. And I think you have to stay inside of that box that will help, especially to grow a product that will help you. Other product managers know, you know where they oscillating inside of.

Bronson: Yeah. End of the day, the team has to believe in the product, not just believe in growth. They can’t believe in growth for growth sake. Or there’s chasing their tail and not getting anywhere.

Nabeel: Yeah. I mean, at some point in anything, you know, you in any decision in life, you run the, you know, the moral hazard of I can do this, but it conflicts with something that that is more sacred. Yeah. And I think if you don’t describe what is sacred to your company, then you will get growth at any cost. Yeah.

Bronson: No, that’s great insight. Now, I think last year sometime you talked about the four major waves of growth that the Internet has seen. And I know you said there were also some minor ones in there. So people listen to this. Don’t take offense. We know there are some other blips on the radar that we’re not going to mention. These are the four maybe major waves of growth. And I want you to walk us through these. The first one was Google CEO. So talk about that way for a second. What was that one?

Nabeel: Yeah, I can walk through all of them. You know, the the larger point here was just to understand that you’re looking for a viable channel. You know, when you started when Jeremy started to Yelp, he understood that there was optimization to be done on the on Google on Google side. And he did an amazing job with Yelp, making it so that when you type in some venue, it’s going to show up in the first couple results. And it not that SEO doesn’t exist anymore. All of these growth channels still exist. But there was a window of time. There was you know, there was a window of like 18 months, two years, three years where the knowledge hadn’t disseminated enough. And so if you were if you were and Google wasn’t even, you know, it was in a bit of an arms race. And a lot of these things, quite frankly, feel like arms races. And so, yeah, the first one was Google SEO. I think Yelp’s a poster boy example there of folks. A lot of e-commerce sites did it, too, where they optimize against that channel and and just grew enormously. And, you know, I think the the second one that came not after that, which I think you had my friend James Courier on a previous.

Bronson: Great interview, by the way.

Nabeel: So yeah, I mean, that that was that was second wave. Right. So he started he started tickle. They did these online quizzes and so on, so forth in his hack, you know, at the time, along with a bunch of other folks were were online address books, we were at a point where we were starting to store online, address books in Yahoo! Mail and in their mail clients. And we had easy export for my address books on the web, on our on our desktop. And so you had these great address book importers and then suddenly, no, now we can get all your friends very quickly. I can start to built a very quick friend graph this prior to social networking. And so again there was a year or two where that was an amazing growth channel. And did some people abuse it? Unfortunately, some people did it, some people use it and built sustainable long term businesses. Absolutely. And so that was that path. You know, I probably the most growth I experienced, although I actually did stuff in the first two as well. I did start ups early in the dot com days and took advantage of SEO for sure. But the Facebook app platform was probably the one that really, I think, catalyzed some of this thought process around growth recently. I mean, it really brought together a very tight community for a couple of years and then of course helped drive long term sustainable businesses like Zynga that took advantage of the request channel notification channel when it was still around. And and if the channel and Facebook and the phenomenal growth of Facebook and the fact that you had this graph of friends to say, now I don’t have to do a weird, you know, upload my address book from a we’re like, I know what your graph is. I know who your friends are and, and can we build a product that takes advantage of that and gives you value for that?

Bronson: Absolutely. And then, you know, now even having the the open graph and you also mentioned even the Apple App Store, along with that one kind of is one of the fourth waves. How did those work together? Because I heard you talk about the open graph and the App Store economy and two sides of almost the same coin, it seems like.

Nabeel: Yeah, I think you’re seeing that if you look at the top grossing charts for, say, the Apple App Store, I think you’ll notice that they’re relatively consistent. They change month over month, but certainly not day over day. And that almost all of those players are using a combination of the Apple App Store, the Android App Store and Facebook to drive to drive growth driver attention and ongoing growth. And I think it’s an interesting time because all three of those factions are really five. Waiting for developers attention. Right now, everyone is thinking about the Apple App Store and how to get in the charts and so and so forth. And so we’ve got games being played and so on and so forth. But but Facebook doesn’t want to be left out of that party. And, you know, they were everybody’s da everyone was building for the Facebook app platform a couple of years ago. If they stop thinking about the Facebook graph, then over time Facebook becomes less and less important. And a phrase I sometimes use is they stop becoming the party and they start becoming a party. I think that’s inevitably going to happen anyway. This window where that means Facebook does different things with their product, it means that they’re trying to find ways to showcase the right types of content. And if you happen to have the right type of content, you’re going to get showcased in the App Store. I mean, in the Facebook graph that’s going to help you drive more installs or more retention back to your product, which is going to drive up the charts, which help you get started, which as we know, starts a flywheel of growth.

Bronson: Yeah, it seems like Facebook is in a tough spot because on one hand they want to be kingmakers. They want to be the people that can bestow major growth on companies because then that allows more developers to develop on the platform, which keeps that virtuous cycle going and they still become the party. At the same time, they don’t want to become too much of a kingmaker because it seems like from the outside at least, that they don’t want anyone to take their open ground up and own it beside them. So, you know, I look at things like video or social cam and it seems like they allowed it to happen a little bit and then they kind of close it down. Do you feel like Facebook has that tension because they know they need to do both in a sense.

Nabeel: And I think a core value. You know, to go back to an earlier comment about core values, I think a core value for Facebook is to, you know, break things, build fast break things. And I think, honestly, a lot of the machinations that outside folks think are well structured thought processes around how to manage apps in the app platform. And so so far on a day to day basis, our for Facebook folks are largely looking at the next goal posts. You know, we have something else we’re focused on right now and we’re aiming there. And I think the key for understanding Facebook is trying to grok where their next goal post is and aligning yourself with those goals. And there are windows of time in which that happens, social impact on themselves. And one of those windows Facebook has changed over time, though. And so like any growth tactic, like it is, it is short term. Short term can mean a week, which would be really unfortunate. Short term can mean a year or two, but they’re probably not permanent. The only thing that’s permanent, that doesn’t mean they’re not viable. Almost every company that has grown like even offline, like, you know, McDonald’s takes advantage of an asymmetry in the marketplace where you can’t get good food when you’re driving across the country because highways are bought by everyone takes advantage of short term window there. There is an exceptional reason why this company can grow when normally it’s not it’s not normal for a company in a span of four or five years to grow to hundreds of millions of users and hundreds of millions in revenue, like no matter it, there’s some exceptions. So I think it’s okay to understand that it’s relatively short term, as long as you know that at the end of the day the product has long term value, that that growth channel might shrink in 12 months or 18 months or two years. And you should be nervous about that. You should be looking for other ways you’re going to grow. Ultimately, the more retention you have, the more user love you have. I can use that firm, the easier it’s going to be to transition out of that particular channel, that area where you’re driving growth and just have users who just love and use your product.

Bronson: Yeah, you know, I think that hits on one of maybe the high level philosophies that I get from you is I read your stuff online, this idea that, you know, entrepreneurship is the exception. So when we look at a graph, don’t look at where it bottoms out and start pointing fingers, look over spiked and say how awesome we got to be a part of that while it happened. And we got to optimize for those moments in time where this was possible. Is that the way you see Autumn Leadership? Just enjoy the high times, expect the low times and optimize for when there’s windows of opportunity.

Nabeel: Yeah, I think the hard thing is to most people, first of all. Absolutely. Yeah. I mean, I started five companies and had the joy of watching take on a public sell to want to sell in business. One hit the wall 100 miles an hour and God, you know, these things happen. And over a journey like that, you I think the the risk that folks who haven’t been through that journey a couple of times do is that they hit very early success and they think they figured it out, that they they’ve learned that permanent lesson. That’s why it spiked. And and same thing. If they fail, they take it as a crisis of faith that they’re just not. They almost you almost read too much into the highs and the lows when on a week to week, month to month basis, what I think makes it more sustainable for the people that I know that have gone back to the well again and again. And over time you see them really, you know, consistently delivering. Those people have found ways to make it a learning process. You know, those people have found ways to learn the little lessons from the stuff that isn’t even high and low where there’s certainly things take away. But they also know there are nuggets in the week to week and day to day. And that’s why I focus on process so much and talk about process, because I think this really is how are you running your company and developing things? It turns out that almost every tactic that you use in developing good growth can be pointed at anything like it is just about, you know, evidence based, but it’s about learning to be more in touch with your customer so you can drive results. It’s the nature of startups that growth is. Growth cures all, and growth is our core need. So we can call this growth hacking because yeah, I talked to 100 guys, 97 of them would like a little more growth and the last three maybe need revenue. And the truth is that you use the exact same tactic to drive higher revenue. You call this revenue hacking, cost growth hacking, you call it retention hacking. It’s about driving results. At the end of the day.

Bronson: That’s great. I love the idea they just put out there that growth is a process because I want to break it down just to make sure everyone’s clear that when growth is a process, you’re in the process of growth, even when the charts not going up into the right, like you’re actively learning and doing things that will lead to sustainable growth even when it doesn’t look good. Because that’s what processes mean is that today is not an indicator of forever is just an indicator of today and I have to learn from it. So I think that is a huge takeaway right there. Now, we talked about the major ways of kind of the past. Do you see anything else on the horizon? You’re in a situation where, you know, you have deal flow. You see what people are thinking and doing that, you know, as normal people don’t get to see what’s the next thing where you have a home slide. You know, there might be something opening up there that might see open for some window of time. Is there anything that you see like that?

Nabeel: You know, I it’s fairly askew trying to give the answers to the back of the book and say, go look at Accenture, try and talk about how people can find that next thing even better. And so I think for me, you were looking for ecosystems that are developing that haven’t matured, right. If you go back to the start up paradigm, the reason people care about a thing called first mover advantage, which is this idea that you’re the first in the market and so you have some advantage over the other people. The reason people ascribe to that over the years is because when the rules haven’t been written, that’s when a startup has the best chance to rewrite the rules, right? Like if the rules are all in place, then trying to break in is hard, right? But Best Buy is not a growth channel because the end caps, how you talk to a buyer like everything about that relationship has been completely set up. Right. It’s been it’s been normalized over time. If it’s been normalized over time, then the big fat company, who knows all that process is going to win. If you’re three weeks after the Facebook platform is launched, no one knows what’s going to happen. And so you’ve got a chance if you’re learning faster, iterating faster, if you’re thinking about it more like if you’re internalizing both the mistakes and the good things that happen, you have a you have a chance. And so when I look at the ecosystem right now, certainly mobile is high growth. If I look at ecosystem, some inside of that that are growing, you look at high growth startups like Pinterest or Lala or Instagram, all of those are their own ecosystems. And do you have some way of grafting on to that? I’m I’m so surprised that there hasn’t yet been a story of two or three startups that has achieved hypergrowth by finding a way to integrate with Pinterest in an interesting way, considering the growth trajectory that Pinterest is on. And I do know several startups. We have one a portfolio that is experience, massive growth thanks to Instagram. So it happens. And so that’s what I look for. I look for other opportunities where you see a huge community forming, where you see people gathering that might have affinity to your product where the rules haven’t been written yet.

Bronson: Yeah, no, that’s great. We’ve had so many people come on and say, you know, they’re zero day hacks. You have to be ready when the opportunity arises. But no one’s actually fleshed it out as well as you just did. No one’s actually talked about the indicators, you know, for for us to sense like, okay, big ecosystems, still chaotic, growing a lot rules aren’t written. We’ll find a way to you know integrate that is incredible playbook. So thanks for walking us through that. Now, you know, earlier you mentioned, you know, there’s those companies that they have. Early traction, and they read too much in the numbers and they think they figured it out. The numbers are always so hard to actually figure out. I mean, I have to ask myself. I look at my numbers and I’m confused by them as much as I’m not confused by them. Right. I mean, that’s just the nature of it. And so I want to ask you about you know, you mentioned something called the caffeine hit of traffic. You know, when a company has a spike of traffic, it’s a boost of caffeine. But if you don’t see the caffeine, you know, kind of crash and you just see the chart chop off right there, you’re like, oh, my gosh, hockey stick. They’re killing it, you know? And then you fast forward a month and it, you know, it drowns and it goes into nothing. So let me ask you about some of the things that you mentioned before. You talked about daily active users and monthly active users. How do you use those or do you use those indicators to kind of tell if there’s some long term stuff happening or if it’s just a caffeine hit? Is there any way to really tell?

Nabeel: So you’re you’re walking through the hypothesis of something’s going up. Another, huh? Feeling pretty good. I just don’t know. How long is it going to last?

Bronson: Yeah. Yeah, absolutely.

Nabeel: I mean, this is where you just need. You need Excel spreadsheets and you need math. I mean, this is where modeling helps. This is where sensitivity analysis, this really gets in a data analysis because otherwise you just can’t know. And you’re certainly not going to run a simple like linear regression and know the answer at the end of the day. I look at DU and MSU and the inversion of that, just for folks who don’t know is, you know, Deja Vu Over. MSU gives you a very coarse number that tells you how many of your daily active how many of your monthly active users are showing up every day. That is only good as a very rough test. Like I don’t necessarily do that if I’m working on product or working on growth on a day to day basis. That’s not enough detail and I’ll get into the detail in just a second. Due you over mewe is still valuable when I’m having a first conversation with an entrepreneur and especially there are products who don’t need high value ratios. There might be. For instance, I don’t think that Volkswagen’s website has a high value over money ratio. People don’t screw there every day, but they’re probably spending a lot of money when they finally do it. And Volkswagen Successful Corporation. It’s not that everything is to have high engagement, but especially in my experience for consumer Internet products, they really only succeed if they’re top of mind. And and it’s very hard to pay top of mind if it’s not a product you use on a regular basis. But I found that when I look across 40 or 50 products on the consumer Internet site, the ones that you use every day are the ones that have a higher likelihood for use tomorrow and the next day and the next day, which is ultimately the easiest way to stay in people’s top of mind, right? A product I use once a year, they’re probably going to forget the brand. It comes time to use it. Yeah.

Bronson: You mentioned day one retention and week one retention. Is that what you’re getting at there?

Nabeel: I think D one and D seven are, which is not week one is different. It’s not did they come back the entire week? It’s specifically a slice of time. They come back on day seven, which is different again, because I’m optimizing and reading for those kinds of statistics. You’re you’re trying to read for is this a daily active users? It’s somebody who will be engaged in this product every day if you want to push more on that. You look at things like sessions per day, like is this a product that the first day somebody installs it, they’re coming back to it three, four or five times that day, which sounds like a very high threshold. But yeah, we live in a noisy world with, you know, there’s your iPad and your iPhone and there’s HBO and Netflix and there’s everything in the world. And if you can’t find a way to break through that noise and be top of mind on that, then you should be working on that if you haven’t found it yet. What I do use, you know, your original question was that caffeine spike and how do you read that caffeine spike? And the there I think it’s important to do old school funnel analysis to know that, you know, I have a sample population that has not been. Hopefully you have you’ve held back 5% of your or 2% of your users that are not seeing whatever it is it’s causing your growth. Mm hmm. So you have a control, and then inside that 98% that are helping you drive amazing growth right now, you’re looking at that and saying, well, why does it grow? Well, it’s growing because it looks like it’s spreading here. They click, here they come, here they do this. Right. That’s the basic flow. And the way you figure out how long it’s going to last is pretty quickly by looking at decay rates. I mean, this is where you just get into real value modeling and see if it’s targeting modeling. All of these things, everything has some measure of decay and and it’s trying to measure that decay rate over time. And you can’t do it by just doing a top line, you know, linear math because, you know, it looks like your monthly active users are going up and they look. GREGORY But if you if you dig in two levels, you’ll find that, you know, you’re you’re bouncing. Or you’re seatrade or the percentage of your debut that shares something to a friend or some, you know, some bit there is probably decaying. And then the real question is, you know, is that today, tomorrow, a year from now. That’s because it just start decaying. Did it start accelerating McCain this week versus last week? Those are the kinds of numbers. Again, they’re not telling you what to do with your product, but they’re telling you where there’s a problem for you to try and dig in and be creative and solve.

Bronson: Yeah. So kind of at a high level, tell me I’m wrong when you see things going up into the right. Figure out what’s not going up into the right or what’s going up into the right at a slower rate. And then dig in because you might be finding the heart of what’s going to be the reality soon enough. What would that be fair.

Nabeel: As a much better way to put it.

Bronson: I want to break it down for our audience because I love the actual math, but I want to make sure that they kind of see the high level as well, because I wish we could do more of the math. I mean, I want you to start pulling out whiteboards and drawing equations. I mean, that’s what I want, you know? So I love it. Now, you’ve seen so many metrics across so many companies that you probably have a sense for what a good metric is in terms of, hey, this is their attention and that’s good for us as a company and this is good for an app that’s free. Like you just have that kind of knowledge. A lot of you know us, we come to it. All we know is our own numbers. We haven’t, you know, been exposed to these other kind of companies. Is there any kind of guidelines you can give us? Like, look, if you’re in this kind of business and you have this kind of number, that’s a good thing. Pat yourself on the back. Like, is there anything like that just to tell us that we’re on the right track, on the wrong track, and, you know, to share some of your knowledge with all the companies you see.

Nabeel: Yeah, I think benchmarking is a systemic problem in the industry. And I’m I have a couple of park related projects that I’m working on to try and solve this because I think it is one of the biggest barriers. Ultimately, I think there’s the one I’ll give you is if we talk about the one for most consumer products, you make a random consumer product and you put it out there on the Internet. It’s probably going to get D1. And D1 is just the number of users today that come back tomorrow. And then it started to come back D one in the next 24 hour cycle. It that will be roughly in the 10 to 20% range for like average product. And and as you know, it’s really like you’re looking for exceptions like, you know, an average that’s not going to and, you know, the really outlier cases, the really amazing cases is you’ll see one north of 50% are sometimes approaching. I’ve seen D1 in the 80% range. Do you want above 50%? Doesn’t mean you have a nice Facebook, but it does mean you have something that’s highly engaging. And it means you certainly should dig in to why to going back, trying to understand that whole process. It’s a wonderful, wonderful sign. And I would say that’s 5% case that you actually have a do you want above 50% in general, if you don’t have benchmarks, the best thing you do have is is change over time. Right. So even if you don’t know whether another SaaS company has X percentage of D1 or retention or click through rate or whatever, you know what yours is. And you can always try and push and move that number and just look at the rate of change. My personal tactic for actually getting benchmarking is I always put together a small community of startups that on whatever I was working on, that we had agreed to share a certain amount of data with each other to help each other, and that takes a certain amount of time and effort. But man, it picked back, you know, back in those social gaming days, there was a group of us, James Currier was a part of it actually, where we were all running social gaming companies. We knew well enough to know that it was about expanding the whole ocean, not about us taking, you know, beating each other up in the middle of it. And so, you know, we would get beers once every six, seven weeks in there, about five or six CEOs. And we just whiteboard out like what’s working and what’s not working. And and, you know, I was lucky enough to have those. I did that even back in the late nineties with some other content companies that were similar to the ones that I was, the one I was running at the time on the web. So I would just suggest that as well, you need to form a small cabal. It doesn’t take a lot three or four other guys in the space and just look at each other’s numbers. That’s not going to tell you unless one of those guys really rocketing. It’s not going to tell you what the best case scenario is, but it is going to tell you if you’re anomalous with a norm. Right? So at D one case, if you’re suddenly staring at 5% D one, a lot of the problem in growth is just knowing which metric you want to focus on the first place, right? You got the one number of decent number of ETR rate share ability. You know, all they’re showing them is, look, it’s like, well, which one of these is actually far enough outside the norm? That’s what that process can give you. It can. At least give you with two or three friends and looking across similar metrics, you can go, Wow, I’m a lot lower on that than I thought I was supposed to be, so maybe I should spend some time there or at least get it up to par, which is probably not even, you know, exponential growth curve, but will at least get me, you know, to be in the game. Yeah.

Bronson: No, that’s great advice. Thank you. And even, you know, the exact stars, you know, the over 50% for D1 and just, you know, that’s the kind of numbers like people don’t hear like that. For most people. That’s the first time they’ve ever heard that. Do you want over 50% is meaningful. And so now they’re thinking, okay, what’s my big one? Let me go look. And now for the first time, they’re going to realize where they’re at in that big ballpark because they don’t have, you know, DC knowledge. Now, before becoming a VC, you had various products that you’ve built. Many of them reached over 50 million users. And you say in your blog you had a few failures as well. You’ve kind of seen it all, given your background, which is unique because you’re an engineer and you’re a designer. So you kind of have these technical skills and these creative skills which are rare. Has this mixture helped you grow products or was just kind of a nice thing that’s a novel novelty about you?

Nabeel: I think the fact that I have a very strong right brain and and or moderately, I would say equally strong right and left brain. I will not ascribe overall value to all of them, but let’s just say they fight pretty evenly on a day to day basis. I think that’s a key to being good at product, right? At least for the way that I do and love to do products. And you probably just sense it in our conversation here today. You know, I do believe it comes and starts from a creative place. The best ideas do and they can be informed by data, they can be informed by a conversation with the customer, or they can be informed in a multitude of ways. But eventually you’ve got to have this core conviction and idea that you should build this, this next thing. But then I think it’s intellectually dishonest to not be critical off the back of that about where you screwed up. And there is no room for superstition in what we do. And to simply say that, well, that guy’s just got great gut, you know, like, I just there’s no there’s no faith in religion and product building, right? Like this is you should look this squarely in the eye and try and learn about it and make it better. And I think art school actually informed that if I, you know, I got computer science degree and I went to art school for design and I think that experience was really humbling. You know, the environment that’s in an art school is, you know, it’s a given Monday, you’re in some class and they say, you know, go spend a week in your studio, dig deep in your soul, find some bit of your past that you don’t really want to talk about, get it up there on canvas, and then you’re going to come back a week later and stand in front of everybody, your peers, and they’re going to rip it apart, not because they want to put you down and so on, but because they want to make you better at what you’re doing. And that peer review process, which is also very similar to science, that peer review process is what I’ve always tried to indoctrinate the companies that I’ve run that I’ve consulted and invested in and talked to. Is this product culture where it is okay to believe that somebody has conviction in something and let them go do it? And the gimme for that is, at the end of the day, I don’t care if the CEO, the VP of product, the creative director, you’re you have to stand in front of somebody two weeks later and say, okay, so we launched a new homepage. I thought that it was going to decrease bounce rate by 25%. It increased bounce rate by 5%. Here’s why I think that happened. And then have a kind of intellectual honesty in the culture that someone else can say, Hey, man, it’s cool that it dropped. You know, it’s cool that you got by 5%. We’re going to fix it. But I actually don’t believe that you understand why it happened. I think actually the conclusion you reached was wrong. Let’s talk about that so that the next version is better. That’s the conversation that you like. If there’s any goal, that’s the goal, right? The goal is to have that conversation. Those conversations are the gold that allows the next week to be better than last week.

Bronson: Yeah. You know, it seems like another theme that’s kind of emerged from this conversation is take what you thought you knew and then go one layer deeper to really understand something. You know, we talked about the caffeine hit, go deeper, see what’s not growing. Talk about the home page, the balance. Right. You think, you know, go deeper. Why did happen? It’s almost like there’s people who, you know, try to hack growth at a service level, but then it’s the ability to say, but why? But how? And going one layer deeper and then really learning something meaningful. So to see that, you know, so many of the questions that you’ve answer now, you guide so many start ups, you know, day to day in what you do. So you see their misunderstandings. You see the the young, wide eyed entrepreneurs and their thoughts on growth. What are some of the misunderstandings that companies have? Have about growth or maybe even specifically when they have a little bit of traction, but they haven’t really grown yet. What are some of the things you just find yourself having to tell them kind of over and over? Like, Actually, this is probably true, not that.

Nabeel: I think. That one of the big problem trying to think through like this week’s meetings with with startup money when I find myself repeating over and over again, I think when it comes to growth are the things that I resonate back to a lot are that it is. It’s a little less simple, but it is a process and not a trick. And it’s about culture, not about a channel. So as much as I talk about the Facebook channel and CEO and there are windows where hypergrowth and maybe they’re not as hypergrowth, now you can still attain growth in any of those channels. It won’t be as easy are the entrenched players are now spreading guidebooks and teaching their VP or using their director of products. But you’re teaching like the all that’s happening. You can achieve growth anywhere. I think what I counsel against a lot is folks actually jumping for the shiny penny like, Oh, so it looks like Facebook is growing this month. Like I’m on Facebook. And I think that’s important because you can sometimes find real anomalies there. But there was a phrase I used earlier that I think is really hard to explain, but I will I try to keep in mind, which is it’s four channels that work for your product in your business, like Pinterest may be a great place to grow a company right now. There aren’t a lot of people doing it. I think there’s a lot of room there. If you’re the right kind of company and there are there are just there. You have to understand A, B and B, inward looking that maybe, you know, your specific consumer educational learning platform is an awesome map or Pinterest is a growth channel. And and so what I see from folks who are young is they have such a hunger to learn that they go and they read what they expect to be the guidebook. And the truth is that what most entrepreneurs do, myself included, is they repeat whatever it is they had as a path to success. They tell you what that was presuming that repeating the exact same path will drive the exact same results. Which is ridiculous. Of course it’s ridiculous, but we do it all the time. I think it’s an emotional Tennessee, intellectual, Tennessee. We do it all the time. And so being able to say, okay, I’m internalizing what this mentor is saying, this investor is saying, this blog post is saying, I’m going to internalize it. I’m not following it verbatim. I’m trying to ask like, why did that work for them? And then how can they twist that and make it work for me, which might be an entirely different way. Yeah. And so that kind of like it’s a little bit of the back of the answers, the book kind of thing, but that desire to grasp quickly onto the hot thing instead of asking whether that thing matters to them, is probably the biggest gap.

Bronson: Yeah. So as an entrepreneur, don’t look for the guide. You are the guide. You just haven’t written it yet because you got to discover it.

Nabeel: Yeah.

Bronson: So I’ve also now, you know, earlier we talked about how the waves of growth, you know, Google and SEO all the way through the, you know, the open graph and that kind of stuff. And there’s always like known channels. I mean, you know, everyone their brother, they know about, hey, you know, here’s how Airbnb use Craigslist. Like, it’s not a secret. Everybody knows some of these channels. Do you have any portfolio companies that are just growing, you know, in a surprising way that are doing it in the most ridiculous way? This was like, wait a second, that’s not supposed to work. Or We didn’t know it could work. It’s not the open graph. It’s like some other thing that nobody knows about. Do you see any portfolio companies like that?

Nabeel: You know? Tumblr. Grew we I know why people retained in Tumblr like we Tumblr went through a actually a good first year of Tumblr’s life. You know, it’s now a top ten website on the Internet and and so on. And, you know, but but I knew David fairly early, and Spark was a seed investor and investor all the way through every round at Tumblr. And, you know, that took a year to really start growing, you know, growing a little bit. But it really took over a year for it to hit exponential growth. And and the way it really started growing in the end was when blogging was introduced. And I think that’s it is hard to grok from the outside in why blogging would lead to growth. And it led to growth in a very, you know, because it led to growth in a very indirect way. You know, it’s not like they found a channel. The way that we talk about it in this conversation, it’s not like they found that they were going to craft off Twitter or Facebook or, you know, some other channel. It was that people wanted to use this product. They love this product. It had such beautiful design sense. And David had such an amazing sense of what he wanted this product to be, but they didn’t know what to write like. Just had a sidebar problem, like, what do I write tomorrow? And so it manifests itself not in how viral I was, how much I was spreading it. It manifests itself in retention numbers. My idea you over me how many times I’m my coming today to read new content. Well, that’s indicative of how much new content there is and how much I have to write. And so blogging for that product allowed somebody to come in and one click say, Yup, that’s good enough. I would have written it, but somebody else said, I’m going to put it on my site. And it changed the interaction model with Tumblr in a way that materially affected its outcome over time.

Bronson: Yeah, so Tumblr users were actually curators, not bloggers, and the blogging just enabled the dormant thing they wanted to do, which is why, like you said, on the outside looking in, it’s like blogging. How does it matter? But from the inside looking out because they’re curators and they just didn’t have a reason to use a product until was introduced. It allowed them to do it. That’s awesome because now product features become their own growth channel. You know, in a way, if you really hit on something core to the user experience of the users. So that’s awesome. Now, you know.

Nabeel: I think people forget when they talk about growth, which is of course, like when people think about growth, they usually think about new user acquisition. And I think it’s important to talk about what you’re trying to get is due tomorrow and go tomorrow. Me could be a current user or a new user, you know, and it’s peak, you know, when we sold to when I sold my startup to Zynga, we had year at the time one of the top ten best growing games on Facebook. You know that at that game we were driving tons of messages every day out to people’s friends for various reasons. And and about 50% of those messages were honed and tuned to be aimed at existing users and not new users. It was about driving growth tomorrow, but it’s a it’s you know, it’s a lot easier to get back at user than it is to get a new user. So you should focus on it and don’t forget it. And to like if you want users to really love, love, love your product, they have to have been using it for a little while. And so by driving that retention, you also drive that user love, that desire to keep using the product which can help you in lots of other ways. I mean, there also the propensity to actually be viral and get new users, higher revenue potential higher, you know, word of mouth higher. Like there’s a lot of good things that come off of off of retention. And when I think about growth hacking in a larger sense, I usually do think about that as a tactic that that process is 50 to 60% about retention and for only 40 to 50% of our users.

Bronson: Yeah, that’s great. You know, when you say that, it makes me think of something. I heard that as a product you’re looking for marriage with the users, not a date. And so you’re looking to really retain them because it’s in a marriage that they will help you be viral, they’ll support you, encourage you, you know, inform you all those things. The people that come and bounce, they help the numbers in the short term, but long term, they’re not helping you grow your product. Now, you’ve answered this last question a million different ways. It’s it’s a high level question. So you can take it anywhere you want. You can repeat something you’ve already said. How are you want to take it? But what’s the best advice that you have for anyone that’s trying to grow a startup?

Nabeel: Yeah, yeah. It’s like a good summary.

Bronson: Follow through, and it’s going to be your your fortune cookie Yoda moment.

Nabeel: I think you it’s not going to surprise what I say. You know, I think you look inwardly, not outwardly. If you’re trying to find path to growth, the answer isn’t a trick on a blog post that you just. Showing up or watching an interview here or going to a growth conference. All of those things are valuable in order for you to build a culture of growth inside of your company. And if you don’t have a proxy for that, if you’ve never been a part of that, try and think about other analogies and use with your team, other analogies that you think will resonate. And that’s why I go back to analogies, like think about what a scientist go through. Think about Einstein is go go through to make a massive breakthrough. Think about what an artist goes through in an art school when they’re trying to become a better artist and they’re trying to go through that process. And that’s the best way for me to give nuggets to you that hopefully will lead you. Hopefully you’re ten times smarter than me. And you don’t use analogies like that to actually craft better processes. And I’ve even developed and make it your own with your own team. I’m very you know, when I sit in it advising people, I say, like, you know, after talking with me for a couple of months and me working with you on growth, like you don’t get a certificate at the end of this year, you’re not supposed to. I’m not giving you a Bible or a rule book. Like these are all guideposts along the way for you to find your own way.

Bronson: That’s great. I love that first sentence you said is like a perfect fortune cookie. Look, inwardly, not outwardly for growth. It’s also on the bill. This has been an incredible interview. I know that people are just going to love this. They’re going to take away so many things. You obviously have seen growth. You know, they’re as an entrepreneur, as a CEO doing it. You get to watch it as a VC and you have a lot to to bring to the conversation. So thank you again for coming on the program.

Nabeel: I love the service you guys are offering. I love what you’re doing. So keep it up. Thanks.