Episodes

Sean Oliver

Sean Oliver

Sean is a product marketer working at Optimizely and living in San Francisco.

He was born in a small town in northeast PA, a.k.a. “NEPA” (pronounced nee-pa). In 2012, the population of his hometown was 1,582. Since then, He mostly lived in bigger towns. He currently live in San Francisco.

Sean tells us how Optimizely actually grew such a massive user base, and he teaches us how product marketing and growth hacking intersect.

TOPIC SEAN COVERS

  • How Optimizely grew such a massive user base
  • How product marketing and growth hacking intersect
  • How does he see being different to grow the business for a young tech company like Optimize versus an older tech company like Microsoft
  • What are growth hacking and marketing techniques over from Microsoft to LinkedIn to optimize
  • What is the product marketer
  • What does he see as next year’s optimize
  • His task experience he dislike most but continues to do because it works
  • And a whole lot more

LINKS & RESOURCES

WATCH THE INTERVIEW

video
play-sharp-fill

READ THE TRANSCRIPTION

Joana: Hi, Joanna. We’ve hear from copy hackers. I am guest hosting on growth hacker t.v. today and I have Shawn Oliver a head of product marketing at optimize lee here to talk with me for a little bit. Shawn, thanks for agreeing to chat.

Sean: I thanks for having me. I’m super excited.

Joana: Yeah, cool. Me too. Okay. So you are in a very enviable sort of position. You’re head of product marketing at the Ultra Cool Company Optimizer, which is, you know, perhaps the best known split testing solution or platform that’s out there. Hopefully any competitors who are listening aren’t like, Oh, but it’s a really well known solution right now. Before Optimizer you were at LinkedIn, which is also pretty cool. You were behind that. Who is viewed my profile feature, which I personally adore. I use it all the time. Right, very cool. Yeah. So you’ve worked with some really highly admired companies that have, you know, experienced this sort of hypergrowth, LinkedIn and Optimizer being those two big examples. But you’ve also worked at that long standing success story known as Microsoft. So having been with if I can just like jump right into my first question for you and having been with these businesses in various stages from, you know, what feels like just the beginning for optimize early today to the very established world of Microsoft. I’m wondering right off the bat, if you can tell us, based on your experience, on what you’ve seen, how do you see it being different to grow the business for a young tech company like Optimize versus an older tech company like Microsoft?

Sean: Sure. It’s actually it’s something I’ve thought a lot about, sort of especially more recently since I’ve been kind of reflecting on, you know, the companies that I’ve worked on in my career and and realize that I’ve kind of been getting progressively smaller as a as it sort of advanced. I think one thing and one thing about Microsoft is that sort of big growth decisions need to be made really thoughtfully and in optimizing. What I’ve noticed and also at LinkedIn to a lesser extent is that bigger growth decisions need to be made very quickly because the market is changing so quickly. And and I think the challenge that companies like optimize, lay or smaller companies that are sort of kind of crossing the chasm, as it were, in terms of kind of getting getting market share and penetration are generally not I don’t have the luxury of being taking their time and sort of thinking about things and making making really thoughtful decisions. Not that we don’t make thoughtful decisions, but I think we need to make sure that it’s most important that we’re that we’re timely and that we move fast. And so I think that’s a lot of where growth hacking kind of comes in. Is that the advantage of growth hacking is that is that you can move quickly and iterate and and test and see what happens and then all things out kind of progressively rather than sort of from the ivory tower, kind of like making decisions and sitting on your know, huge customer base, which is which is a lot of what Microsoft does. I think Microsoft’s in a position where large growth won’t really happen anymore. And just for a lot of their core products, because they’ve reached such a significant level of market penetration that there aren’t that many more people to to sort of grow to. So it’s really about driving engagement and sort of maintaining their market position.

Joana: Okay, that’s interesting. Now our next question on the same, you know, train of thought is around, you know, I wanted to learn about growth hacking techniques that you’ve seen carried over across, you know, the different organizations you’ve been with. But it sounds like maybe Microsoft didn’t have as many. I mean, growth hacking as a term tends to be rather new as well. Yeah, I mean, that the concept is so. But I am wondering, you know, what, if any growth hacking and marketing techniques have you seen carry over from Microsoft to LinkedIn to optimize? I mean, what are. I don’t know. I was wondering, what are the big players doing that the little guys should think about? And maybe in the big player space, you might want to focus more on LinkedIn and optimizing that if Microsoft isn’t doing that much.

Sean: Yeah, I think that’s a good question. And I think it’s it’s not necessarily the case that Microsoft doesn’t do growth hacking. I think in in my specific area, there probably wasn’t as much of that. I worked on the Windows business, which is Xbox Software. It’s it’s a much more established, sort of slower moving kind of space that that’s changing with Microsoft that I’ve seen some of the recent announcements. But I think they’re they’re getting to a point where they’re a little bit more connected in terms of like more frequent releases and and having a more dynamic product. But in general, what I’ve found is that since moving to the web and I think this is also true of the Web areas within Microsoft, so like the Bing team, for example, or the the Microsoft store, like Microsoft.com, things like that, because those areas are much more sort of iterative in their process because they have the advantage. Webb Yeah, I think one thing I’ve noticed that’s sort of consistent across all of the companies that I’ve worked at is this idea of contextual marketing and personalization or sort of understanding using data or different signals about your customers, and then leveraging that to deliver sort of relevant experiences or exciting experiences. That’s something that I’ve seen. I’ve seen the team at Microsoft do a really good job of with some of their they’re sort of like social proof. They have this really neat integration with Facebook where they can, you know, generate search results that are relevant to you based on sort of what your what your kind of friends are sort of clicking on what they’re looking at. And I think that’s really interesting. I think that LinkedIn is probably the king of this, at least, at least in terms of the company that I work with or obviously other social. They do a better job. But I think LinkedIn has done just in time democratizing data and information within the organization to the extent that even the marketing team is fully fluent in the sequel and other sort of languages like Python and JavaScript, and everyone is kind of getting involved in the data culture and using data to make decisions, data that I think is really powerful.

Joana: That’s that’s incredible. I had no idea that marketers knew that kind of I mean, that’s impressive, right? I mean, it says a lot, obviously. Yeah, right. So you can have those conversations even that marketers traditionally haven’t been able to have with the programing team. Right. And then, of course, when they’re come up with creative ways to grow using the data. Yeah, that’s very interesting. I want to hear more about it, more like examples and things. So we’re going to get into though as hopefully. But first, before we even go there, I think it’d be great to maybe get out kind of an understanding of product marketing. So you’re heading up product marketing at Optimize. And it seems to me that tech companies have shifted a bit in the last five, six, seven years from these teams of marketers, which sat kind of independently from the programmers to product marketer. So can you tell me what is what is a product marketer?

Sean: That’s a really good question and one that I think I have been asked probably 100 times.

Joana: That’s a really common question.

Sean: I think. I think that it’s a really it’s product marketing is definitely an evolving discipline. It’s something that is is a slightly different company to company. But what I found just in general, the product marketer is really the product expert kind of on the marketing team. And what that means is, is product marketers are they have a very deep understanding of the product, the features that are on what the benefits are of the product to the, to the customer. And then they have an even deeper understanding of who that target audience is, who that customer is, or that consumer is, depending on your business and and sort of what their needs are. And then I guess the whole idea of product marketing is about figuring out what the right set of products benefits are, that map to those needs and then determining kind of the value proposition and the positioning of the products, the messaging of the product. So just how we explain the product, how we talk about it, and then also how we bring it to market. And I think that that involves not only the words we use, but also the channels we use. And so I think there’s a lot of interesting product marketing that happens in product, but also a lot of interesting product marketing that happens through other channels like like online marketing and events marketing sort of display or things like that. So I think there’s there’s sort of a it’s a little bit of a mixture of things. It’s kind of like I like to think about the product marketing as sort of the outbound kind of manager of the product. And then the product manager is sort of the inbound focused person. So product management is really driving the strategic direction of the product itself. Like what features does it need in order to achieve this sort of vision or this customer need? And then the product marketer is doing the same thing, but just around how we communicate the product, like what does the product need to communicate or what does it need to to do in terms of a customer engagement perspective? So I think those are sort of the ways that anybody can.

Joana: That’s interesting. And, you know, as you’ve been talking, I’ve been thinking that a lot of the marketers that I hope none of them are listening, but a lot of the marketers that I’ve worked with might be really uncomfortable in this sort of like world where they’re so closely tied to the product. It hasn’t like in larger tech companies, obviously. I think though in the startup world it’s a little bit different and I’m thinking of a few of the startup founders that I know who are programmers, like they’re trained programmers, but they really enjoy the marketing side of it and they’ve found themselves kind of pulling away from wanting to build and maintain a product to wanting to, you know, keep connected to it, but do all that marketing and bring it have the go to market strategy in in their hands and to experience that sort of thing. So it’s interesting. I know it’s not directly tied to what you’re saying, but product marketers. But you’ve brought up that thought for me, which I think a lot of startups probably could appreciate a lot of the strategy.

Sean: Yeah. And I think I think what we’re seeing sort of at least definitely in the startup space, I think more recently like there’s sort of been a wave of kind of the designer as being kind of a really important person in the startup space. And I think that now we’re kind of seeing the marketer also becoming becoming more and more important, or at least the discipline of marketing specifically. And I think that that that’s sort of a natural kind of evolution as like startups and sort of products become a little bit more saturated there. There’s there’s more need for marketing, right? There’s more need for understanding your customer and understanding their needs and how. Kind of like cut through the clutter like a better term. Like I don’t really like to use that expression, but that’s that’s sort of how I would describe it. Yeah. And, and I think I just want all of the kind of innovation that there’s been around marketing recently. And I think growth hacking has actually been a really positive sort of movement that sort of helped elevate. Marketing is kind of part of the I end of the conversation with the product. I think that I think that it’s it’s all sort of like marketing is becoming more more science and art. It’s still it’s still very, very art and science, but it’s sort of shifting more in that scientific direction. And I think I think all of these tools that we’ve been developing have been have been helpful in that.

Joana: So yeah. And amen to that completely, right. This movement toward more of a science and an art which some people find, you know, problematic or like that’s your point. But but I think that people who are in the growth hacking world are take great comfort in that and find that it’s true. Right. So I’m wondering, I mean, we do still keep the term product marketer separate from growth hacker not and that might not be for any reason other than people don’t have titles called Growth Hacker in organizations. A lot do, but some don’t. But what what do you say is the difference or even just the overlap, even between growth hacking and product marketing? Are they essentially the same thing or what makes them not exactly the same thing?

Sean: That’s a that’s a great question. I think that I think that the difference is can be really tactical because I do think there’s a lot of overlap. I guess I’ll talk about the overlap first. I think that the where where they overlap is the growth hackers goal in many cases. And this is just based on my observations. I’m sure many people will have different, different opinions on this, but it seems like the idea is to drive either either user growth or engagement, right? So like, I think driving more users into the product or driving more engagement with it once you once you have those users in one way or another, sort of trying to direct behavior in that in the way that is sort of best for the product and also best for users. And I think that can be a big output of the work that product marketing does. I think product marketing can function in that role. And I think I think the best product marketers are the ones who do that. But I think that you can separate them out and say sort of talking about the differences. Now, I think the product marketers are are sort of further up in the funnel in terms of driving that that sort of marketing strategy. So what that what the value proposition is how to do the like what the messaging should be. And that’s all informed by this sort of customer understanding and this customer research. And I think the growth hacker is then that person who is sort of taking that and kind of turning that into action through through different kind of in product levers. I think that I think that there are other levers available such as display ads or PPC or blog stuff like white papers, things like that. Like there’s obviously lots of like marketing channels that exist. And I think I don’t think a growth hacking necessarily is a channel, but I do think it’s sort of a a it can be it can be sort of like a tactical execution of some of these things, ultimately to drive the goal of user growth or user engagement, which is which I think is a shared goal of product marketing. So I really think it’s best when when they’re the same person or when they’re on the same team. But I can understand how like they can be different. The skill set can be slightly different. There’s, there’s obviously more, a more technical understanding needed to do, to do some of the growth hacking things. But in general, I think that marketing is moving to becoming a more technical discipline. And I think that marketers who resist that are probably going to find themselves in trouble at some point, because I don’t think that we’re getting I don’t think we’re getting less technical by any means. I don’t think traditional marketing has as much of a place in this new economy. So I think it’s it’s we need to sort of as as marketers kind of change with the times and be and be more more comfortable working with data and more comfortable working with engineers and with product managers. And and I think that’s I think that’s that’s sort of where the whole industry is going. So I’m excited.

Joana: But that’s one big sound bite and I love it and I want to like quote it everywhere. I think it’s great we’re not getting the last technical right. That alone says everything.

Sean: Absolutely.

Joana: Yeah. So that really, I think, is a good way for me to now ask you some of the questions. I’ve been kind of dying to ask you about growth hacking tactics almost. But so in your experience across all these places, optimizing LinkedIn, Microsoft, I’d like to know and I hope others would like to know what’s been your favorite, you know, growth hack that you’ve witnessed or that you’ve been directly involved in?

Sean: Okay, that’s a good question. So I think that one of the one of the coolest and most interesting things that we did actually actually at LinkedIn, LinkedIn did a lot of this sort of iteration. They have a lot of first party data about their users and. I think it’s really cool. I won’t give away all the details about this, but I’ll sort of talk about it because it’s pretty exciting. There is there’s a page on LinkedIn called People You May Know which many people who use the product are familiar with. It allows you to. It uses an algorithm to determine people that you you probably know based on certain social signals. And and what that page does is it services other members on LinkedIn for you to connect with. So one thing that we started to do at LinkedIn while I was there was was interleave or sort of intersperse nonmembers on LinkedIn with the members of LinkedIn or people you may know. So if you go to the page today, you’ll see you go to that page. And for the most part you’ll see people email who we’re also members of LinkedIn, but you may also see people who are not members, but who you also may know. And that’s also based on sort of social signals. You know, perhaps you’ve imported your your contact list in the past or maybe you have connections, you’ve done that. And and so all that information is available and it’s being used for your benefit to connect you with them, help you build your network. But it’s also helping LinkedIn grow by it, by making it sort of more frictionless and more intentional around how you how you employ your contacts. So I think that was that was one of the coolest sort of experiences going to have at LinkedIn in terms of in terms of like a growth there. There are some epic growth hacks that LinkedIn has done in a day that I wasn’t necessarily involved with directly, but that’s probably the coolest one that I can think of in recent times.

Joana: That’s really cool. Very smart, right? Some smart people there. Yeah, that’s very smart. So so you were directly involved in this. And what has what did you see as the outcome or what were the results or how did it affect growth?

Sean: Yeah, so I think so. It doesn’t have a positive impact on growth. I can’t share the exact numbers. Yeah, the trading company and everything. But I think that it’s it had it had a very positive like the the pros of what we did. There were we had we had some customer or customers and some, some members at LinkedIn who were, I think, less comfortable inviting friends who weren’t on LinkedIn into their networks because people don’t want to be spammy, you know, and we don’t want to just like import their entire contact list and spam all their friends. But there’s something that’s that’s very, I think, smart about the intentionality of what we were doing with interleaving. I think it gives it gives users the ability to just select a single person that that they know that they feel comfortable with inviting. And it does it sort of in the same area of the of the site where you would do this other very related action and just connecting to people who are on LinkedIn. And so I think it’s it’s actually pretty cool because it sort of removes the friction of having to ask you to go into this different mindset of now you’re not you’re inviting somebody outside of the network like they’re all in your your network, right? They’re all in your, your sort of sphere of of connections. It’s just a matter of whether or not they’re a member of LinkedIn. And I think it it should be that same sort of comfort level with inviting them. And I think that was one of the smartest things about this, is that is that it kind of tapped into that and took advantage of it.

Joana: Yeah, definitely. So what would you say? I mean, I’m thinking through how you can use that other people. How can you take that idea and apply it? But what do you think the average, you know, growth hacker person listening right now can take away from that and like go and deal with that if anything?

Sean: Yeah, I think that I think that it really it’s obviously that’s a very LinkedIn specific example. Right. And one thing that I really took away with it is that is that there are there are ways that you can sort of group kind of similar or complementary actions together and have them be kind of in a series. I think that, you know, users, especially if you have existing users on your before or members or customers or whatever who are using your product, if they’re going to get used to kind of doing certain actions and or you’re going to create a an experience that enables some some action. And I think if you can tie those actions to complementary actions, I think you’re you’re better off. So an example of this is that this is this is sort of a very different example, but I think it relates to the same thing that optimize early on. We we recently tested this, adding this call to action link on our results page. So when you when you have an experiment and a test experiment that’s winning and statistically significant, we have a link that asks you if you want to submit a customer story. And I think that’s a really interesting idea because it sort of gets at the same thing. It’s not exactly the same thing at all, but it’s it’s it’s interesting because it’s a it’s an action that’s related to the experience that you’re currently sort of participating in. Right? Like you’ve got a result that is that is successful and maybe you want to share it. Right. And so that that helps us because we get a. Customer story that we can share with potential customers. But it also helps you because you can promote your business and win that you’ve just gotten. And so I think it’s it’s sort of a really thoughtful it’s about kind of making making thoughtful decisions about servicing actions. Yeah.

Joana: Yeah. I mean, some of the, you know, more technical growth hacks out there are always like great to hear about, right? But then there are these kinds of things where it’s still meant to grow. Right. You’re still doing something that will lead to growth in these different ways with content marketing, right? You need to create content. So it’s not the same as the other example you gave, of course, but it’s really it’s clever to write it smarter than saying, okay, well, we as a business really would like to get more case studies on our blog. Well, let’s send out an email asking for case studies. And it’s like, why is our response rate low? Well, it’s because nobody’s thinking about it over there. Like what? They’re in their inbox. But when they’re there seeing the win and feeling really good about it and, you know, feel good is a very persuasive sort of technique, right? Or whatever the word is I can’t think of in my head, but getting able to feel good is persuasive. And so if you’re in a positive, emotional state, you’re more likely to respond well to a request. So I think it’s very bright. Yeah.

Sean: Yeah.

Joana: Good takeaway. Good thing to go right and to look in your look at what you need to get done as a marketing team and see how you can stop relying maybe on email quite so much as some people would do and put that more in. When we talk about in App, it gets me thinking a lot about optimize. Again, who has recently added in the ability to do in-app testing, which I think is brilliant, right? Like great. Thank you. That’s really good. People need to work on that. So I mean, optimizing is really one of my favorite company is, you know, you’ve got this really simple, beautiful solution to a problem. Every online business has done all that fantastic. And of course, even with a lot of other split testing platforms out there, it’s kind of like optimized has become the default, the one people talk about when they’re talking about B testing the first. When you think about when you were a consultant talking to a client, you say, do you have a testing platform set up or No, you don’t. And everybody just knows to go straight to optimize the right, you kind of become the default without, you know, losing your trademark, which I think is great. Famously, you guys have had presidential candidates use your solution, right? So, I mean, who is next on your acquisition list after presidential candidates have used it? But I mean, I really do wonder what does. You kind of up look like we’re optimized. What’s next, given that you’re in this kind of earlier stage, but you’ve got so much success happening already? What does what’s what’s next? What do you see as next year optimize?

Sean: Yeah, I think that I think that one thing we’ve really noticed with sort of looking at kind of how our customers are using the product and specifically looking at a bar more kind of advanced customers and the things that they tend to do. And and sort of also just thinking about how the product, the market is moving. I think where I mentioned earlier that it seems like we’re moving toward this era of sort of more contextual marketing or a more personalized sort of marketing. And I think the advantage of or the ability to do that is really is really predicated on the acquisition of data and the ability to to leverage data and turn that data into action. And I think one thing that Optimizer sort of makes really accessible to two marketing teams or to two teams that are perhaps less technical or less or have less access to data is a sort of democratizes that and makes it accessible broadly. And so I think as we think about next steps, I think we we’re kind of thinking about ways that we can enable more of that sort of personalized sort of experience delivery. I think that some of our smarter, not smarter, but in some of our more more savvy kind of customers are are using optimize in really interesting ways. They’re using it to deliver personalized experiences. And we’re seeing that increasingly, increasingly, more often where people will be setting up campaigns that are targeted toward a select set of users for for us based on some piece of information that they’ve collected about them through optimizing. And then they’re delivering that, that, that experience. And so different, different customers are seeing different things. And I think I think that’s sort of the next wave of like, you know, how can we make this more powerful and more automated for customers? And that’s that’s sort of what we’re thinking about in the future.

Joana: And that’s really interesting when we’re talking about personalization, essentially, right? Yeah. Yeah. So when I have seen personalization initiatives, they’ve all been they require a lot of content, right? You do an AB test, you need two pieces of content that you’re testing against each other. You write a website without testing anything. You just need to create one piece of content. You personalize for different segments, and now you have a lot of different content to create. Do you have any sense for where how you might help people get over that objection to personalization, which is how am I going to write for different web pages for these four different segments?

Sean: Yeah, I think that I think that actually any testing can sort of be the the the answer to that, not necessarily the complete answer, but I think that’s how that’s how we’ve seen it kind of have its entry point. I think the way you mentioned earlier is really true when you when you’re creating an ad test, if it’s truly just a control versus like variance test, you’re really only creating one extra set of content that you’re testing. And I think that’s enough to kind of get you started, right? Yeah. I think running an HIV test will show you whether or not there are certain segments or certain audiences that that outperform the control or the mean. And I think the the interesting sort of insight there is if there if there isn’t a segment that outperforms like perhaps there’s there’s the test is inconclusive or perhaps the meeting is just at a certain level. And there are no there are no like outlier audiences there. Maybe you don’t have enough audiences to do personalization. Like there are some sites that have they have not a lot of traffic in there and they aren’t and they’re very niche, very targeted. And so there isn’t really a need for that. And I don’t think it’s necessarily right for every business, but I think for for the vast majority of large businesses, certainly like there is, there’s a lot of different types of people I know and no individual person is sort of being the same. And so I think that they’ll typically find that there’s some profile or some demographic is outperforming. And I think it just takes that one sort of proof point, that one kind of data point to, to show that there’s something there to kind of instill that in a culture, or at least to bring that into it, into the into the conversation.

Joana: Right. And so when I hear about personalization, I think a lot about the problems with AB testing and multivariate testing, obviously requiring a lot of traffic to get to a point where you can get that confidence that you need. So on and so forth. All the problems that come with trying to ab test when you’re, you know not Amazon yet but so the hope for me kind of has been well maybe with personalization businesses that have less traffic and get more done, but it’s kind of sounding like you’re still going to need a good amount of traffic to use personalization effectively. Because you’re still going to be learning a whole bunch, need a lot of traffic coming through. Is that accurate?

Sean: I think I think there’s a way to think about it. Is is I don’t think that’s entirely accurate. I think it depends on it really depends on the business and it depends on the profile of the customers who are coming through. Like in general, the way we the way we sort of like we work with a professor at Stanford who studies and focuses on statistics. And he’s been and he’s been helping us kind of like articulate are kind of statistical methodologies and improves kind of the way that we we calculate statistical significance and statistical power. And I think one thing I’ve learned about this is that there’s there’s a difference between sort of statistical significance and statistical power. And they’re both important. I think statistical power is where that volume kind of comes in. But but you don’t need necessarily huge volume in order to be statistically significant. So there could be if you have two drastically different sort of segments of population where they are serving, like I think of examples like, I mean there’s there’s, there’s very different like left as one is coming to mind left has their driver audience in their in their customer like they’re they’re by your audience they’re what they’ve done a really good job of is having sort of differentiated content that focuses for them. That’s a really obvious example. But I think there there are other examples where you can see even with a small amount of traffic having a statistically significant result, that also has sufficient power to be to be reliable that you could that you could go for. So I think it it depends on you probably won’t see like let’s say you’re a site that has 5000 uniques per month, you probably won’t see a statistically significant result for a point, a 1% improvement rate, whereas LinkedIn or Amazon or Facebook could easily see a statistically significant result for that level of improvement. But that’s a pretty small level of improvement. I probably wouldn’t make that big of a difference for a business anyway, given the size. Again, it depends on, you know, how valuable each visitor is. But I think that in general, when it comes to personalization, if the personalization is good, the difference should be very big and and it should be meaningful. And the live two should also should showcase that. And I think that’s something that’s really important to keep in mind as we think about this.

Joana: Yeah, very cool. The difference between even just significance and power, I haven’t really heard that much about and I work in the optimization world and I haven’t heard much about. Do you guys have anything on your blog about that by chance?

Sean: If you recently posted a blog article about it and there’s also a link in our our knowledge base that sort of goes into into a lot of detail on that. I can actually I can share it with you.

Joana: Yeah, I think a lot of people would be interested in that because I think the testing world has been so focused on, you know, you post a case study on a test that she ran and immediately every other consultant on the face of the planet comes to your post and says, like, show me the data. Where is your significance? Right. And so we’ve been so focused on that, that confidence level and everything that goes with that, that it’s good to hear about different ways to approach that question.

Sean: Absolutely.

Joana: So, yeah. Okay, very cool. I’m going to I’m going to look for it to and help people listening who are testing look for it as well. Yeah. Yeah. So I’m wondering. Okay, you write optimize, we optimize like does testing helps other businesses test? And of course, tests for itself, right. Is is testing its own stuff. So along the same sort of lines, which, you know, two or three that you know of, I know you’re relatively new to optimizing, but which two or three tactics or techniques or even just one like crazy unexpected tactic have you do you know that optimizer used in its early days and then continuous to use to move forward? Like what is the one thing that they did before that they could never give up doing? What could optimizing not grow without doing if it came down to like one thing or two or three.

Sean: That’s that’s a great that’s a great question. Can there’s one that definitely comes to mind and we talk about it a lot. If you go to our home page, optimize e-comm, you’ll see you’ll see our logo, obviously our tagline and a a URL address bar with a will to try it out. But we tested out button. That was a very early growth hack that our co-founders, Dan and Pete did back in the day. And we continue to do to this day. It allows customers to enter a URL right into our range or our home page without having to sign up or create an account or give us any information. And then it puts them right into the product and allows them to start setting up an AB test and making variations and changing content on the site. I think what’s really clever about that and what’s really powerful about that, other than the fact that it just showcases and allows the product to speak for itself and showcases how simple and sort of magical the experience can be. I think it also it also builds a lot of trust and value with customers. So I think it. It gives those customers the the the ability to start kind of developing something that’s valuable to them before they sign up. I think we’re not asking them for information about themselves until they absolutely need it. Yes. And I think that’s a really important thing to keep in mind. I don’t think it makes sense necessarily unless unless your product is not is not sufficiently valuable. I think that you should always be putting off the sign up until until it’s absolutely necessary. And so, you know, I think that I think that if a customer creates a variation using our Web editor and and hasn’t signed up and wants to save it and try it out and actually reuse it, and that’s when they sign up. And there’s there’s a need for them to do that. And I think that’s that’s a really important sort of insight that they tapped in on early. And we’ve actually been we definitely would never explore sort of removing that. I think one thing that we’ve done, if anything, is is highlighted even more and make it more prominent as part of our experience.

Joana: That’s interesting. And I know, right. It’s a lot of SAS apps could do that, right. If you’re an online solution, you should be able to do that. It takes a little extra work, of course, to build that if you’re just trying to like get your product up door or something. But. But given the success that you guys have seen it, so do you have any sense for how it has helped? That tactic has helped optimize early grow so that people can say like, okay, well, it might take more time or it might make us rethink our cart our whole way to get people to pay. But it’s worth it because it works optimizing, so maybe it’ll work for us. Do you have a sense for how it’s worked?

Sean: Yeah, I do. I think I think there’s something very inviting and sort of like this is all kind of qualitative on my on my perspective. We may be tested it heavily, but the the the thing that the insight I think that I hear the most often is it makes it it sort of reinforces the fact that the product is is powerful and also easy to use because we’re just kind of giving it away. And I think that I think that we’re not we’re not trying to like it. When you come to optimize it, you don’t get a marketing pitch, you don’t get a sales pitch, you get you get the product, and then you get to try it out. Like, we’re not in the business of selling you certain services and selling you like marketing, which is we’re in the business of selling in this product that is going to help improve your business. And I think that’s sort of the underlying message with that. And that’s when, when, when we test sort of removing that. And it doesn’t work out in terms of sign ups, I think I think that’s why the customers don’t know necessarily what they’re getting when they’re just clicking on a button. But when they’re when they’re entering their euro and they’re actually experimenting with their site, they can see the value and they can experience it. Also, AB testing is sort of this is kind of a different a different but related point is that AB testing is is not necessarily a kind of a like in the growth hacking community. I think it’s a very commonly used tactic. But I think that it’s something that that not many more mature businesses are familiar with. And I think putting putting customers at larger companies, look, we have a lot of enterprise customers who are coming from a more traditional marketing background who may not be as familiar with it, need to see what it’s like before they can invest in it. And I think that’s that’s another sort of byproduct of the fact that we’ve just made this thing so easy to use. The customers can just walk in and try it.

Joana: Yeah, definitely. Yeah. Super smart. Yeah. So I’m wondering then I really want to know, I guess also like, these are all really great and I should be taking notes, but I’ll watch it later and take notes. Any in product growth hacks that you guys have done that you’d be willing to talk about or share?

Sean: So there are some okay, some that we probably shouldn’t talk about.

Joana: Those are the ones you should talk about.

Sean: So some things that we’ve done that have been pretty cool. One one is sort of when we recently launched a an iOS sort of a B testing product. It’s in beta, but it’s called optimized for iOS. If you if you have a nice app, check it out. What we did in product was we we noticed that there were a lot of customers who were running experiments on their mobile website app. And so what we did was we surfaced a, a sort of guided tour link into a, into the iOS product experience from just from within that sort of app testing experience. If you’re, if you’re setting up an experiment or looking at results for an experiment for a mobile website, you know, odds are you may, you may have or be considering or working at a company that has an iOS app as well. And so we we wanted to make that sort of message available in product. I think there are some other things that we’re doing that are that are sort of more and more sort of socially triggered. So if you have teams that are working together, you know, there there are ways that we can in the past we’ve sort of like directed customers to. Seeing experiments there that are sort of doing well that maybe they weren’t familiar with. I think there’s there’s a lot of under the hood stuff that we that we’ve kind of done over the years as well. I probably shouldn’t talk about now, but I think that I think the one thing that’s really cool about optimizing is, you know, you can like we’re always experimenting and and I think I think I think that we’re trying to kind of live the ethos of what we preach, which is which is that experimentation is good and using data is good. And and I think it’s it’s been it’s been going well for us.

Joana: It seems to have been. Yeah, no kidding. So you’re talking about obviously experimentation and data. I know a lot of people who are trying to grow or even just test come at things like, oh, let’s sit around and brainstorm what we should do. Is that something that is part of your process for developing like growth, new growth hacks or new ideas? What you can do or how do you guys come about new ideas for what you can work on?

Sean: So that’s a that’s a good question. I think it’s there are some situations where will where we’ll brainstorm things. I think that more what’s really interesting about optimize like so let me actually talk about some of my past experiences first because I think that’s helpful to have the comparison at certainly in Microsoft’s mindset. LinkedIn, to a lesser extent, we would have brainstorming sessions where we talk about what we’d want to experiment with in terms of growth and optimize. I think that we’re in this interesting position where we are a smaller company, so that might have something to do with it. But I think that more and more broadly, like we, we have a very rigorous sort of experimentation and kind of growth hacking culture here. And so we’re always kind of talking about different ideas or different things that can be can change that we can that we can experiment with. I think that comes up in a lot of meetings that we have. It comes up in casual conversations. And one of the great things about working at a company is it builds a product that makes it easy to do this is that we can we can have a conversation in the morning about a hack that might be interesting to try and then we can have it implemented and roll it out and see how it goes. I think it’s it’s just a really it’s a really exciting time from that perspective. I think that there aren’t a lot of controls as to what we can or can’t test in the product as long as it doesn’t like totally break things. But even if it does like, we can, you know, turn it off.

Joana: So so yeah, definitely. Yeah. That’s very enviable, right. That you can that’s you’re in that position where you can come up with an idea and then go.

Sean: Yeah, I don’t.

Joana: Think it’s.

Sean: Brutally unique to us either. I think that companies that are able to maintain this, this sort of culture, of sort of transparency or at least the democratization of data and making decisions about data, I think I think that if companies can embrace that and understand that there will be some times when things break. But for the most part, like, you know, everyone’s trying to do the right thing for the company. If there’s that trust there. I think that, you know, the the the output will be better. I think that people will be doing things that are more exciting and just more more interesting and probably more more or less expected. More unexpected.

Joana: Yeah, harder for the enterprise level organizations. Largely to do what? At least there I found that when working with enterprise.

Sean: Licenses, I think that that’s I think that that’s in practice that seems to be the case. But I think I’ve actually seen examples of enterprises too that have been able to embrace this using, optimizing. I think it really matters what the tools are and what the what the the culture is about the tools like how I think it needs to start, especially in enterprise organization, which tend to be pretty sort of hierarchical and structured, it really needs to come from the top down. There needs to be executive level or senior level kind of approval of this type of thing. But I think that those you know, those are the types of companies that I think are the ones that are really pushing the envelope. And those are probably the ones that are more exciting to workforce and employee perspective, too. Hmm.

Joana: Interesting. Okay. Well, that’s cool. You kind of hinted that, you know, based on what optimized the clients are doing about some cool things people are doing to grow their businesses. So if you can share anything, you might not you don’t have to name names if you can. But have you seen any clients of optimize like do anything very cool or get some very interesting results with their own, you know, growth hacking initiatives or even just like any test that they’ve run, whether on their website or, you know, in app.

Sean: Yeah. So I can give you two examples. One, I will, I will. I can give the name of the published customer story that I really like. And the other one I’m going to try to not give the name is secretive about it. So the first is code.org, which is a a nonprofit that’s that has this goal of sort of extending computer science education around the country. They they try to get basically they try to teach kids to learn how to code and and create sort of the next the next age of growth factors, which is awesome. And. Engineers. And they think one thing that they did, which was really surprising to me, just because the lift was so big, was they tested two different sort of headline and call to action combinations on their assignment page, one that was much longer and that that sort of emphasized this aspect of community and one that was shorter and more functional and was really descriptive about what code word was. And they found that the longer one actually generated a 29% improvement in signups.

Joana: We have to pause because I’m a copywriter. So all the copywriters of the world are like clapping now, like saying, okay, go ahead.

Sean: It’s huge. I’ve never seen a headline test yield that big of a result before. It was so exciting. I think that it really got to the fact that that their their mission and sort of like they’re they’re they’re sort of the community that they develop is so important that it really just compelled their audience to sign up. And it actually resulted in that 29% that have translated to 18 million more registrations. So 18 million more kids around the country learning to code.

Joana: Amazing. Okay. Very cool. Wow.

Sean: And so and so those are 8 million.

Joana: Well to stick with.

Sean: For getting there. So the other one sorry, I can’t give much fewer details about, but I think it’s really, really neat is we have an e-commerce customer who tested the impact of using our kind of BlueKai segments. We talked about we talked about personalization a little bit in this talk. BlueKai is a data management platform that was recently acquired by Oracle. It’s sort of an enterprise grade search solution, and this was an enterprise customer. But basically what they do is they they collect third party data. So they they drop a cookie on customers to sort of when they’re, I guess, a browser and clients. And then and then as you go to different websites that are also in this BlueKai network, they collect kind of data about you and then you can use that for marketing perspective and targeting. And I think there are there are sort of less expensive alternatives to this for for those like smaller startups out there. But I think one thing that was really interesting was we had a customer who was sort of a a they didn’t have a very big kind of e-commerce presence on their website, but they were they were kind of growing that. And I think one thing they found was that among new customers coming in, they really didn’t know who they were. They didn’t know anything about them. And what they were able to do with BlueKai data was they could they could set up optimize the set of segments targeting, targeting or optimizing variations targeting BlueKai segments. So they could say, okay, I know that this person is in the market for home goods, or I know that this person is in the market for X or Y. And then they tested the impact of showing their generic sort of sale that they were doing every season with sort of differentiated kind of sales that were that were specific to their differentiated sale, promotional banners that were specific to the product category that they were interested in. So they actually didn’t test any messaging change. They only tested the imagery on the banner. And what they showed, they showed different product sort of pictures based on who they were and they saw a very big left. I can’t talk exactly about the numbers for that, but they saw a very big list from doing that. And it was interesting because they were able to test it based on individual segments of customers coming in. So I think it’s just a testament to sort of the power of data, but also like an interesting story about third party data relative to first party data. I think a lot of companies nowadays, especially smaller companies, are really interested in gathering first party data, and that’s great when you can do that, but it’s also really hard to do. And so there are there are advantages to having access to third party data as well, but it is really cool.

Joana: Big advantages from the sounds of it. Yeah, yeah. That’s so cool. Yeah. I want to hear the number, but I know you can’t it.

Sean: So yeah.

Joana: It’s a good story though, right? And a good lesson there, too. Very cool. Okay. Well, I’ve learned tons from you. Thank you. Yeah. And I would just have, you know, this one other question that I like to ask because yeah, it seems like it usually leads to something that can be a little interesting as a final note. So as a product marketer, Shawn, what task in your experience do you dislike most but continue to do because it works?

Sean: That is a really good question. I as so I do like a lot of things that I do. I think that the one thing that I probably don’t like as much is, is sort of the the kind of events, marketing sort of aspect of product marketing and just of marketing in general. The reason is because events marketing is. If you’ve ever sort of participated in events before or gone to one or, you know, a sponsored one, they’re very expensive and they take a lot of coordination and there’s always a chance that people won’t show up or that the right people won’t show up or whatever. And and even if they do that, it won’t yield sort of good results. They’re difficult to track. But. But I think that I keep like what one thing we have here that’s awesome is we actually have a person who does does an amazing job in marketing and and what she does really well and why why I think we continue to do it or participate in events is is there are ways that you can kind of hack event marketing to make it really sort of scalable and that you can make it really measurable. So that’s using things like I think that if especially for younger start ups, like if you’re thinking of participating in an event or sponsoring an event, a mistake that I see a lot of startups make is they just sort of like set up a banner and a booth and they kind of go there with t shirts and whatever, and then they hand things out and they try to get the brand out there. There’s definitely value in that, but you can’t measure it and you can’t take action on it. And I think that there are ways through through in-person sort of promo codes or sign ups on site and and through sort of recording things like actually interviewing people who come to your booth and then giving them something for that or or maybe asking for feedback or having people complete surveys. There’s so many ways that you can get value out of those events that I think are, that are, that are just really clever and and it really important to do if you if you want to sort of have it have kind of a scalable growth kind of way of doing that. It’s something that I’m still like on it’s definitely not a an area of expertize for me, but it’s something that I that’s one area where I think I’ve learned the most since then, since I started here is, you know, how can you make those really impactful?

Joana: So I didn’t expect that to be what you’re going to say. That’s really interesting. I mean, event marketing or being involved in these events. So what would you say is like a goal that people should? So I’m on balance. On balance is running their first conference. Remember, they have an events person as well dedicated. What should what’s what would their goal be or what would what is the goal of the average event? Like what? I mean, it’s hard to say because of the different types of events. Right.

Sean: But as I say, so for a conference, if you’re targeting your existing customer base, we actually just did our first user conference back in April and it was really successful. It was actually way more successful than we thought it would be in terms of people who who showed up and who attended. It was more than 200% attendance, which is great. I think one thing that’s that’s really important is to be targeting existing customers is really to make sure that you’re sort of evangelizing the products with them and giving them the tools they need to kind of be evangelists on your behalf. And so one thing that we did when we when we had our customer conference this year was we made available sort of promo codes for them to share or kind of optimize with with their their colleagues and peers. And that that actually worked really well in terms of sort of our customer generation thing. We also we also did Net Promoter Score surveys. And so I think NPS is is a number that sort of discussed a lot. And I think that it’s it’s depending on how you ask it. And when you ask it, it can be extremely biased, but it’s a really good indication of it’s probably the best indication that I can think of in terms of customer service satisfaction and customer kind of loyalty. And and what we were able to do was do a pre and post service survey, pre service before they even had heard about the event. And then post after like a month or so after they had attended. So the effect of the event, it kind of went off in terms of the higher. Yeah, and I think that that’s another way to just sort of gauge value. I think the one, one thing that I that I think about a lot more is sort of like participation in events that aren’t necessarily targeting a customer base because those can be really valuable too. And I think it’s much easier to measure the value of those things. So like participation in industry conferences, there are definitely ways to use that to sort of generate signups and to generate just sort of like, you know, net new kind of leads or net new sort of customers who may not have been, you know, in your in your existing sphere of influence before. And so I think that it’s it’s it’s a little bit of art and science. I think we talked about that before. But I think this is this is one that, you know, is is something that I think there should be more sort of published about event marketing and how to and how to do that effectively.

Joana: Yeah, I think that’s really interesting. And I wonder when you say there are ways to do it. Like, what are some ways? Right. So you say we gave out promo codes. Like how? Like. Promo codes were able to share with their friends. Great idea. How did you do it? Right. And yeah. So maybe we’ll just start with that one. Like, so what do you do to get people to actually. Take your promo code and hopefully then go share it.

Sean: Well, so what we did is with this company, typically we were giving them a promo code and they gave them access to a beta version of our product that we hadn’t released yet. And then that promo code was also shareable within their sort of like brand community for a limited amount of people. That allowed them to kind of like invite other people into this beta version of the product as well. And so I think that it provided value back to that customer because they were getting early access to something that they didn’t that they wouldn’t have gotten if they hadn’t come to the conference. But then it also gave them sort of extended value because they could they could share that and be seen as an influencer among their kind of peer group.

Joana: Okay.

Sean: I think that’s really important.

Joana: And then also then when you’re speaking out, someone else is at an industry event. How do you how do you share? I mean, I know that when I speak at event, there’s usually a, you know, sort of rules around how much you’re allowed to, like, say, about your organization. Right. Like final slide. You can basically put whatever you want on that final slide while you’re taking questions and hope that people like look at it and type it in or whatever, if you have a QR code, whatever you’re doing. But how have you what have been your tactics there when you’re at industry events where others who are watching might be going to? How do you do more with those events? Like what are some of the tactics where you actually get, you know, is it is it handing off business cards? It’s just making sure you have those handy and just like give them out like a, like there candy or something or. Or do you get candy or like, what do you do to what are the tactics to make sure that if you’re at an event, you’re taking time away from building your product or from, you know, sending emails or updating your drip campaign or whatever it is you’re doing. This takes time and money. How do I get the most out of it? What are the tactics?

Sean: That’s a great that’s a great question. And I think it’s important to understand that there aren’t sort of any silver bullets. They’re sort of like they’re what we’ve been Horowitz calls led bullets. So there are a lot of individual little tactics that you can kind of do to to sort of chip away and make sure that you’re that you’re doing a good job of this, I think. And again, I’m not the expert. What I would say is making and making sort of contact information if you’re a speaker in the event making contact information available on every side of your desk. Right. Making your Twitter handle and your email address or whatever it is visible at any time because you never know what you’re going to say that might resonate with somebody and they’re going to want follow up with you on that. I think being visible in the event is also a big deal. Having having a business card available is helpful. But I think one thing that we’ve also found that’s been kind of interesting is if you have a business card available with sort of like some kind of incentive on the card or some sort of thing that kind of like a promo code, for example. Like I hate to keep saying promo code, but I think they actually really work depending on who the audience is, those those things can sort of live on and be and be shared and be reused. And it’s a reason for somebody to pull that business card back out of their pocket or out of their wallet or whatever later. And and look at it again and and perhaps contact you or perhaps just sign up for your product to see if they if they had it before. And so I think that there there are sort of like books like that. I think there’s other things that you can do in terms of kind of like in the most important thing that you can do in terms of like led bullets to keep saying like words. But the best tactic is to just have really compelling, interesting content and to and to leave the audience with something that’s that they haven’t heard before or that or at least a way of thinking about something that they haven’t really considered. And so that really comes from understanding who’s going to the event and how they think about things. And then and then doing a lot of like research and understanding sort of what, what are the, what are the mental models that we can challenge and what are the things that that where the areas where where I can add value, as with based on my experience, that’s the sort of different that they may not have expected. Right. And I think that’s I think that can be sort of the the best tactic of all when it comes to when it comes to that.

Joana: Marketing is very cool. I like it. Good stuff there. That’s great. And you were mentioning contact information. So I mean, I’ve had a great time chatting with you today. And of course, I wonder if you can share how people can follow you or get a hold of you if they’re like if they have a question.

Sean: Yeah, absolutely. So my, my, you can contact me on on Twitter or my my handle is just my name, Shawna or CNN or IVR. And my my email address is Sean Ashton all over me. And he’s.

Joana: Brilliant. That’s awesome. Well, it’s been really enlightening. And, you know, I know you’re relatively new at Optimizer, but it sounds like you’re going to do amazing stuff there. I can’t wait to watch what happens as you’re there. That’s amazing. Cool. Is it? Anything else you want to share? Any closing thoughts you might have?

Sean: No, this is great. I really appreciate the time. I’m very excited, as I said, to be to be part of this. I think this is great.

Joana: Cool. Yeah. I’m really excited for this to go up and be shared with everybody. So thanks a lot, John. It’s been great to meet you. Thanks.

Want to Become a Better Marketer in 5 Minutes?

Yes! I Love to Learn
Are you an entrepreneur who is trying to grow a startup?

Get the strategies, motivation, and in-depth interview with all the details every week!