Win Customers' Hearts With Data—Google's Neil Hoyne Masterclass

I'm excited to talk to my guest today his name is Neil Hoin he's the chief measurement strategist at Google and he's got degrees from Purdue University and UCLA senior fellow at Wharton if that wasn't impressive enough for you he also wrote this book it's called converted the data-driven way to win customers Hearts Neil welcome to the show yeah thank you so much for having me Chris I just want to dive right into it um I'm skimming through the things I can pick up and the first thing that you say is a real person is on the other side of the transaction and you say well this book will teach you how to win their hearts can you explain what that means I mean in the simplest sense you know we become the more we use digital the more we use technology the more we become disconnected from people on the other side and you see this with people that just rely on dashboards and data a lot of what I like to say a lot of head in there a lot of thinking and logic not a lot of heart and so it's not uncommon to see people whether they're entrepreneurs or their marketers at large companies looking at dashboards they just be like look how many page views we had even on social media look how many likes we have and you get disconnected from the story of the people behind it you know the greatest problem that we've had with marketing over the past 25 years at least probably you can make the case 100 years has been that it's often difficult for businesses to understand or believe if it's working the way it's supposed to be you know what would somebody have bought my product anyways if I didn't Market or if somebody's building a brand does that do anything different and because of it there's just been that Focus entirely on metrics and this is just taking it a step further to say look we're going to look at who those people are who those relationships are and how to build something real from them so if we're looking at the analytics and we're not understanding the person behind it what are we supposed to be doing because maybe I'm guilty of this myself I see that something's performing or it's hitting a certain demographic how do I see the person behind the data for a lot of companies honestly it's just giving them the simple Focus to it now I kind of joke with people here you know when they look at analytics and data there's a certain desire for Perfection right I want I want perfect information I want perfect decision making I want to reduce risk to zero the business world doesn't work that way the business World Works you're better than other people now I joke with this with my wife I'm not sure she appreciates a joke but I kind of say to her I'm like look I am probably not the best guy in the world I'm just better than other people you had you know I think he's the best one I found so far and it's very similar with with businesses look somebody doesn't have to have a perfect product or a perfect advertising campaign they simply need to be better than other Alternatives so if the alternatives are companies who are looking at you purely from a dollar and sense perspective purely from a transaction side like did you buy my product yes or no and you have another company say look we care about this relationship we care about the Loyalty you have with us then all of a sudden they're more attractive and so when we talk about understanding a little bit more about those people it's and this is really what the book covers collections of steps things you can do as as a marketer as a business owner as an entrepreneur to at least get one step closer to having a real relationship with those people a real connection and being able to harvest some of those results even if you're not perfect I can see why your wife would not like to hear that statement from you there were there were a few Joe I try to be very I mean you should kind of laugh at it you know I'm married up you know that's just kind of like eventually she's gonna figure it out there's a certain imposter syndrome as there is at Google uh with everyone believing they snuck in it's always the case sometimes in relationships you're like you're really too good for me like when are you gonna figure that out thankfully it hasn't come yet which is fantastic so I I continue just as I tell us a comment I try to be one percent better I try to just be one percent better than everybody else to be like oh you know this works and that's the same thing for companies okay let's continue on that thread let's go deeper into the whole relationship thing and you draw the comparison that this is like dating and and you don't want to be too intense or direct and scare them off you're trying to court someone tell me how because that sounds very human that idea that this might be a long-term marriage or courtship that might and that's what we all want to build as a long-term relationship with our customers how are you looking at this from the lens of data and marketing there's a joke early on in the book where I kind of had this thing where I was like it marketing today is very similar to marketers walking into a bar and proposing to the first person that they say the original version of the book wasn't proposed but in this case proposing to the first person they see and you think about how awkward that would be if somebody did it and you would say well why would anybody do this and you say well look if that marketer is purely looking at their motivations their incentives maybe their goal is to get married that evening there's really not an incentive to pursue anything else because nothing else matters nothing else is counted just like with marketers you only care about sales immediate transactions it doesn't matter if they love you they want to come back and have a longer conversation your boss is going to ask you how many people said yes and so this is how marketers approach it by and large as they go out and they propose to everybody and then they go back at the end of the night and it becomes a game of volume did I propose to 100 people a thousand people eventually at least a data two or three percent of people I don't know who they are we'll say yes and then the question is well how much of an opportunity do you have to improve if that's your goal if that's your approach and and you may be able to change what you say or maybe where you go but there's only so many levers you can pull on now when we look at this from a very human lens you're like this is absurd this is not the way that people behave behave and marketers are slowly realizing you're right this is not the way customers buy either even simple retail purchases often take three or four interactions on a website three or four conversations before they're willing to commit yet marketers are still focused on no I want everyone to commit right now and they struggle and they're like well how come there's not a good relationship there so here's what marketers are doing today marketers are going out and they're trying to get everyone to buy right away you're like hey that that doesn't sound that bad why would that be bad and he's like what if you thought relationships work this way and you're like wow that would be kind of awkward or I'd draw another comparison in the book if you met somebody and then as soon as you leave that bar they start calling you up and sending texts to be like hey did you think about that date do you do what did you think about that proposal what about now and then they keep following you like no no I'm at my parents house now hey what what about and that's that's the same thing we say see with online advertising is that if people start showing you messages as soon as you leave the website display advertising purchase intent their interest in your products your brand actually decreases and again just like in the real world device at least you know earlier on was you generally want to wait two or three days before you call somebody it's very similar to advertising if you wait two or three days before you start sending them ads versus your last conversation it actually improves their responsiveness and so there's a lot we can learn from both sides it's just being able to put them within that context so that way they make sense and we kind of pull ourselves out of that purely data-driven mindset that we've had for the past decade or two okay I think I understand what you're saying and I'm listening also for our typical audience person which is either a solopreneur or an entrepreneur with less than five employees and they they vary in their creative practice and execution of this I'm on board I hear what you're saying it sounds to me like people are trying to close too fast and then they're just looking at the transaction person and they're forgetting that there's a human being and I think in today today's age we have so many sophisticated tools to track people and follow them everywhere we can do email nurture sequences and we're just bombarding them with stuff first of all it's annoying second of all it comes across as a little bit desperate and I think that neediness is it repels people away okay so I understand the concept right and if we continue to to draw from the analogy of how you behave in the real world if you're really attracted to someone what do you need to do and we're not most of us I guess aren't looking for that just that one nightstand but we're actually looking to build deep meaningful relationships with people how do we begin to change this because I'm on board now yeah I get it you're good you're good it's like well it makes a lot of sense when you think about it that way so uh the the first the basic tenants of it number one is exactly what you said the long-term relationship the importance of it the second part of it is also to remember that progress in this area does not mean Perfection it does not mean going out and everybody you talk to loves you it does not mean that every person you meet you will have a great long-term fulfilling relationship with you know when we look at customers across businesses it doesn't matter the size it doesn't matter the industry you always see the separation where a very small portion of customer will contribute a lot of value and a large portion of customers will contribute almost nothing at all and there's nothing necessarily bad you need those people in your life and I compare it very similar to you will have Partners friends family members that you couldn't imagine having a life without and then you'll have the more transactional relationships it'll be someone comes in and at that moment let's say a server at a restaurant an Uber driver that brings you back from the airport that interaction was cordial and there may have been some value exchanged but you're not going to see them again you know they're not you're not going to go to them for advice and so it's just prioritizing to say out of the customers you do have out of the relationships you have which ones have the most potential and this is where this idea comes from and again the metric I feel is terrible I wish they had a better name for it but what we're really talking about here is is looking at a metric like customer lifetime value what is customer lifetime value if you're new to it it's an individual level prediction as to how valuable every relationship will be to you that's exactly what it is the same way that you might be able to look at someone individually and say I really connect with this person I love this person I'm so glad they're in my life this is just doing it except scaling it across hundreds thousands in some cases millions of customers to tell you what those differences are objectively it doesn't require a lot of data it simply requires kind of an understanding of previous interactions how much people spent and when and then just over time you start to get that picture to say these are the people I really need to pay attention to and these are the people I need to avoid and so this gives you just an understanding it's looking at here's all your customers here's who you should focus on and everything that happens after that point just kind of builds from that story which is to say if there's people you really get along with then businesses and we can go into this start asking themselves why what makes these people special how do they like to engage with me what products are they interested in how often do they come back when how did I meet these people and at the same time who are the people I want to stay away from and just like we have those interactions in our own lives where we can kind of look at people and be like I don't know I don't really connect with these types of people they're not my My Tribe you start to see the exact same thing from a business you start to see some people that say look they'll come in whenever I want to offer them so something on a deep discount or if they come in they waste a lot of my time and this is just making it a little bit objective so it's taking that that Heart part of ours that wants to be human and connect with people and bridging it with the mind to say we can use some of this data and just put it into a perspective that's really useful you know I love what you're saying completely resonate with this I need help in understanding the data part so if you don't mind I may ask you some really boring questions because I know the company you work at Google you guys are the masters of data I'm feeling everything you're saying and then I'm just scratching my head thinking okay let's just take for example a company like ours where how we actually are able to finance what we do is because people buy digital products mostly or they join our coaching community so if I were to look at a customer lifetime value I could filter it by who has spent the most I can also look at who is the most engaged and I could see some of those things but how do I do this Beyond building a store in reverse engineering who these people are how they came into our ecosystem are there tools and things that I can use to find out more or do I need to ask them how did you find out about us and do this on a one-to-one level until I have enough data points let's fill in the missing pieces great question so number one is um the models that we're talking about here the way we calculated because people start thinking you're predicting relationships this is a future and this requires lots of data and lots of expensive no it doesn't uh and in fact uh one of the things that I was deliberate about when writing the book was we dropped that chapter out because we didn't want it to be a requirement the value is in the strategy and thinking about the problem what people naturally do the model's already established we've been there for decades I just need to connect the dots what I ended up with on the book was there's a really great company down in La a company retina AI it's their CEO mod fantastic guy and I said mod I said you build these models all day long I have all these book readers that are going to want to calculate it for their small business and their startup can you help them and they were actually pretty generous and they said yeah yeah if anybody you go through the book website you click a button will model your data for you and so you don't need to worry about it we're not going to charge you we're not going to ask you for credit cards there's no upsell there's no subscription it's just we want their value is using that data we'll talk about the the models are just commonplace and really it's connecting the dots and showing people and I joke about this if you go to Google and type how to calculate lifetime value the first three pages are wrong and the run-in circles and you won't see anything out of it so this was the easiest way I said look I'll give you all the math I'll give you the models if you're that type and you want to see how I'm really building this I'll give you all the research so you can see it but if you're busy you know like I just want to see the numbers I want to see who I should pay attention to then we have some of these Partnerships like right now where they said hey we'll just do it for you for free we're not keeping your data we're going to delete your data after 48 hours just genuinely good group of people just to help people through that first step now what do you get out of this so your data goes in and the data comes out very much like a spreadsheet in one column you have here's everybody's name that you send and in the second column here's how valuable they're going to be in the future right so we know what they do in the past but does that spending continue we'll we'll get that prediction and so you have these two columns in the spreadsheet but a lot of people have questions is it engagement that matters or the fact that they engage with us frequently or is it the types of products they engage with us on or is it how we acquired those customers these are legitimate questions just again as we would have in real life is the people that I met here or there or do I get better people with these Hobbies or those hobbies and so just picture that spreadsheet and add a third column so for some business it'll be here let's add a third column here's how I was introduced to that person a fourth column do they participate in this product a fifth column what product did they buy initially and what companies do is they just group their customers and they say how does that behavior change based on where they came from and they start to see differences they start to see well these relationships that came from this channel were really valuable this channel not as much or maybe this campaign or we find really high value customers sometimes buy really cheap products early on they're just getting acquainted with us or it's not uncommon to see sometimes we want people to buy right away but our best customers actually want to be deliberate they want to spend more time on our website learning about our products before they raise their hand and want to be known to us and so now we just get to say we're not combining all the data we're starting with a hypothesis what do we think is a driver of these relationships and then we just add it to the spreadsheet and we group these people together we say does lifetime value differ based on how these people are behaving now I will tell you sometimes it doesn't so I work with some companies and they look well let's take a look at people that are using our mobile app versus not and they're like well they're spending the same amount they're staying around the same amount of time nothing's changing but even that's insightful to say look you have this mobile app you have a permanent place on their device and you're not really building a relationship from it you're simply giving them an alternative Channel then converting over the web and sometimes those are meaningful discussions as well but this is the way that we start to bring together those Dimensions who you get along with best how they're behaving with your business and giving you different actions you could take to improve that value the spreadsheet that you're talking about with the sample columns like how much money they've spent engagement however you want to Define that or like the initial products they purchased are those a pretty good starting point for most companies or does this need to be you have to kind of look at it well this kind of business those kind of columns are not going to matter in this kind of business you need totally different kinds of things I say that the goal is always as we say to be better not perfect every column that you add I say it really starts with two things if you have a legitimate hypothesis about the data how the data matters to you and you have the ability to act on it then it's a fine data set to bring in but I say this because oftentimes I work with companies where they say the starting point is to get all of our data together everything that we have and clean everything up and I say no I say give me the data you have available and just tell me that if I see something different do you have a course of action that you can take in your business to adjust if you know what advertising Channel people are coming from and you have the ability to spend more or less on an advertising Channel or if people are talking to different customer service reps on the phone and you find their lifetime value differs based on that call or the representative they talk to then that matters or you find people from different regions and you can move your marketing and your focus to those regions then great you can do it but if you get into one of those pictures to be like well you know let's see how a conference from two years ago behaved if you can't go back to that conference if you can't reach that audience then what are we looking at this data for and so that's where companies start is is not even the data at all but it's to sit there with almost a blank sheet of paper and to say what do you think leads to Great relationships in your business what data would you use to prove that and then that's what you bring any new tests and you say how closely does that align to what that hypothesis is some companies are surprised I like that hold on one second there seems to be somebody at the door I have to tell my wife to maybe keep it down hold on one second all right all right sorry about that I love that opening question like what date like you start with a hypothesis hypothesis you're looking for some kind of pattern a prediction like we love these kinds of customers what is common about them we start there and we don't have to have all the data points as you said just get going and it can change it can evolve you can add more data points until you can start to see some kind of pattern right I tell people this is my my favorite one is of a camper in a forest and a bear is chasing you and uh you know you think well it's like well what are you going to do to get away from the bear and you can't run faster than the bear and then you realize the problem is you're simply running faster than other campers that's a solution and I'm surprised as to how many companies how many entrepreneurs when they're confronted with the problem it's like all right a bear is chasing you and they're like all right okay we need to do is we need to sit here and think about what's the strategy for out running the bear and then they sit there for a 12 16 months like we need to collect more data to solve this problem it's like no no just just start running and you're fine especially with this economy there's too many companies that are sitting there being like we want to be deliberate and think about this it's like no move faster than the other campers and you win with things especially my my world is online advertising online advertising as you know is an auction you win by having better information being able to make better decisions than your competitors you don't win by having perfect information you just need to be slightly better and when you break this down and you say look really all we're doing is going one step at a time saying who are these valuable people and then one hypothesis of Linux what makes them different the products they buy how they engage it allows you to make a slightly better decision very quickly that advances your business and builds a trust in the approach you're doing and then you can keep doing it as opposed to see some of these strategies like we're going to do machine learning and it's going to take two or three years to implement it's like that that is a risk right you can't do anything for two to three years you have all this Revenue you're passing up hoping for this godsend of a solution to come I'm like no just just take your take your spreadsheet and add a couple attributes and just start doing something to meet slightly better people can you share an example or two of when somebody starts to look at their data figures out who their best customers are who they want to work with and looking at the data they discover something and then what they do with that Discovery so that they can increase the value of the relationship the length of the relationship and ultimately build the best customer for them let's start with that example I mentioned earlier about the mobile app I have worked with a number of companies where they're convinced they're like our mobile app is what's driving deeper loyalty and engagement and you look at the data and you say no these people are spending the same as non-mobile app users and then they've been able to go into their app and say well what exactly is the strategy of your application travel is a great case a couple years ago there were a number of travel apps where the entire Focus was is getting you to book again they weren't talking about loyalty programs they didn't have any value-added Services especially hotels didn't have digital key cards it was all you book or you don't and it was just a choice well this app allows me to book faster and they weren't they were pretty surprised when lifetime value was the same nobody was really bidding but their goals were well we want to drive sales otherwise our app doesn't get any credit and now you give them this lens and you say no if you add this loyalty program information people aren't bidding again but it's bringing them back and that's not necessarily a bad thing you see it on on other websites one of my one of my favorite examples just of what something someone learned this is actually some academic research that I talk about in the book was just around gift giving you know oftentimes especially during the holiday season you see that option to give something as a gift and they found actually in the data that people that were on their very first purchase were buying a product as a gift for somebody else ended up having a higher lifetime value themselves for the next 12 months I guess it has something to do with if you're giving somebody a gift it's a reflection of your taste right and it's like yeah this is these are brands that I love and you feel more closely and connected to that brand so you spend more yourself there's also been cases just with the advertising Channel customers have come in on where there are differences based on where customers are required from on their very first interaction that doesn't translate to how much they spend on their first purchase but they do come back and spend twenty Thirty forty percent more and for those cases what those companies have done is they simply said look we're going to spend a little bit more time and emphasis on these channels because we can now prove their longer term value as opposed to resetting that clock every time someone comes in and buys ones you just reminded me of a conversation that I have with my Chief Operating Officer his name is Ben what Ben does is he's the master of the data set and I'm not so much because the numbers make my head hurt and he'll come up with some hypothesis based on the data he's seen and this is where it gets real interesting with us because the data is the data but the human interpreting the data creates a narrative and that supports the strategy that they want to take and I look at the data and said I have a completely different conclusion based on the exact same data set you just told me right now so now when data turns into humans to interpret and then to make educated guesses how do we know what path to go or do we do both paths or do we split tests or do you have any kind of advice for people people who look at the data and start to form their own narrative I get this a lot but but I understand the the nature of the question I'll give you I'll give you a simple example if uh you know let's say I gather everybody from your side we're all sitting in a room and I say all right here's what we're going to do I'm going to give you an opportunity and let's just say let's say it's a market growth opportunity or new product you could launch and what I'll do just for fun to get things started off is I'll give you the overview of the opportunity so imagine I'll slide across the table a sheet of bullet points five or six and here's the opportunity here's the market size here's what we may price it here's how we build it and I'm gonna just ask you Chris what do you think and you're going to have your opinion I may I miss any of you said Ben I'm gonna send over to Ben what do you think and we do this across a couple hundred people maybe and we're going to get some people that look at the opportunity and just say look I love this and other people that look at this say I'm staying away from this and generally what everyone will say is like we have a lot of we have disagreements here we don't have a conclusion what we need is more data right we need to understand this what is this a real opportunity or not and that's fair enough I'm a data guy I can empathize with people like that so over the next two months I'm going to send you more and more data this is an experiment and I love to play around with things so if you were on that side where you said you love this new product you think it's awesome I'm going to just give you a whole bunch of data that tells you why it's garbage and if you think that the opportunity is terrible I'm going to give you a whole bunch of data to say this is an opportunity you can't miss out on and then we're going to get back together after two months and I'm gonna say how do you feel about this opportunity now and nearly everyone will keep their original decision the only thing that changes is that most people will feel more confident with their original decision because they had time to look at that data and so when people come to me and they say the solution is more data and more analysis I look at it I say is it really is that really going to change your point of view and and here's where the answer a little uh comes into it is that you know these researchers were looking at this at this experiment they said what could we do to actually change somebody's point of view as it turns out it wasn't more data it was simply reminding people when they went into the problem that being human beings we are biased towards our initial Impressions even if they're not publicly declared and we have a very difficult time taking in and processing new information that simple awareness I wasn't giving you Advanced Techniques and well we're going to do meditation in between no just this is what it is made people more open-minded and willing to accept new information and new data and when they did that that simple disclaimer I think it was then after that point a majority of people changed their decision to match what the data showed them and so when I look at that I say those disagreements are very human it's just important for us to recognize that they exist so that we can make slightly again slightly better decisions overall than what we had and I generally find companies come in and that's kind of the the understanding is if I go in there and I tell them just to look at the data the data will help you decide those disagreements are rampant because people see that data in different lenses if I tell them I say look you're going to bring in a whole bunch of biases then people deliberately try to be more open-minded and they're easier to find a consensus around you just make them aware of how they might be approaching this it's almost like you saying some of you might be racist and then you have this discussion it's like okay I'll be a lot more sensitive and listening to this right yeah I mean and that that actually is a known bias in terms of surveys that we talk about briefly in the book about if somebody asks you your age your gender where you grew up beginning of a survey it'll actually change the answers that you give one survey I believe was done one study over at Harvard actually found when they asked I think it was elementary school children middle school children they gave them a math exam and they remind them of their gender at the beginning they said that women actually performed worse on the results of the test just because they were anchored across with a Believer these social differences that never manifest themselves oh but we do fall into those identities and those classifications subconsciously and so that's why it's careful to remember just that's what they always say you put those questions at the end of a surveys because you don't want to bias people by uh by their peer groups by their experiences those should be left out but they are very human and they do impact everything there's even they even said that when you talk about eating habits an interesting thing they said just about Brands and brand awareness there have been studies that say look if they show people ads for McDonald's then their eating habits will change later on in the day now it doesn't necessarily mean that they're going to go to McDonald's it actually means that they found that people will have less healthy eating habits later on the day because they were conditioned by seeing those McDonald ads and you start to think you're like well how is my my brain shifted by it and the larger macro study in that case was they were looking at if it was just simply seeing an advertisement for a particular brand that can change how we respond for the rest of our day what happens when you're driving home and you go past that big fast food restaurant does that change the way that you behave later in the day and those researchers will make the claim that yes in fact it does wow okay this part isn't so intuitive if you didn't tell me this I wouldn't have even guessed that this is the reality where the sequence of the questions changes the way people behave and so putting the demographic information at the end of it will help them answer in a different way that that is just crazy for me it's like okay and as you're talking about seeing ads and affecting your behavior I just flew back from Las Vegas and as I'm going from security to the the terminal where I'm supposed to be at I'm walking past a bunch of fast food places and then all of a sudden it starts to increase my appetite for and I I'm gonna confess I did not make the best I had fried chicken I did not make the best choice for my body yeah I'll say you do that you do it I always I eat McDonald's at the airport it always always that's why why McDonald's I recently introduced my my two and five year old I introduced him to Happy Meals and so a little bit more frequently because they get really amped up about it but you know with with a lot of these things I think what it is is these this is just we like to believe we make objective decisions about our health about how we answer questions if you ask somebody about something be like well how do you make a decision about what you're going to eat they're going to think about it in a very objective perspective well I want to take good care of my body this is what I have a taste for they don't necessarily think about all the different factors even some of the questions um you know reminding people about their financial circumstances so I think there was one study asking people how much money do you have in your bank account or how much credit card debt do you have will change the way that they spend immediately after whether they become more aggressive or more conservative reminding people about when they're going to die or other people dying making people more aggressive with how they spend more risk taking these things happen all the time and again it's not necessarily going back even to that original question around data-driven decision making it's not necessarily that we want to change people we want to just become very robotic and this is how we make decisions but oftentimes people will make better decisions in personal life and in business if they're aware of some of the factors that are driving these changes and it doesn't have to be everything you don't have to uncover every Nuance about human behavior against success with yourself with your business is just knowing a little bit more than where the mass Market sits I'd love to get your perspective on this as I'm listening to you I'm checking in with myself to say okay am I guilty of some of these things how am I doing this and so as a designer as a person who likes some of the science and the data I make a lot of decisions intuitively I have I think a pretty good pulse of what our customers want because I'm transacting not transacting I'm having conversations with them online all the time Non-Stop and so I use that as part of the way I my data points and they're pretty soft and then let's say here's the example we use a program called tubebuddy and tubebuddy says if you feed me the title of your video on YouTube we will score it and we'll tell you if it does well and we'll make suggestions using some kind of algorithm machine learning I don't know how they do it and they kick out a title that scores much higher than the one that was put in and I look at them like I don't think any human is actually going to type that in and we've done some experiments with this because my team will just use that as like word from God here's the Bible it says to use this they put that in and the videos do not perform and I gave it about two days to collect enough data points we pretty much know that the video is or is not going to work within the first 48 hours and then I go back and I change the title using my intuition knowing what it is that I know about our our audience I change the title to something that feels more human and surprise surprise 75 of the time it outperforms the original title Based on data help me or am I just is this a massive form of cognitive bias you know I'm just as you're telling this story I'm like I can't wait to see what it comes up with like this it's like like old data-driven analyst yells at yells at machine learning models uh will be the name of this the video I have a a slight I know I have a a negative bias on my side towards a lot of machine learning lately because I've seen too many companies entrepreneurs say if I want to raise money the easiest way is to claim something's using AI or machine learning and so there's often a difference between what these tools are capable of versus the reality the other part that we look at is very much how human beings can be biased we like to think computers and data are impartial machine learning models are biased based on the data that they've already captured and if the data that they've captured is biased in some way in terms of the age of the data or the types of videos they're collecting then you're going to see it you're also going to see just that propensity unless the the only data source they were pulling out was top tier performers I would almost suggest that they run the risk of being biased towards the average and where you're seeing success is your intuition is saying look this is something that's not reflected in your data set that could be better and they have that joke about machine learning where it's like kind of like a joke to be like if you're a machine learning algorithm and all your friends jumped off a cliff would you jump too and machine learning algorithm will it will go that's what everybody's doing that's what I will do and and so there has to be that purposeful effort but what it goes down to with me is to say I look at anything that's um you know people often ask me they say what's the standards of machine learning or AI in your work and where I generally go with it as I say oftentimes it's impossible for me to understand everything that goes into the models I carry a healthy amount of skepticism with them but I look at them as being almost like a friend or advisor that comes over it'd be kind of like you know curse of yours out to me like Hey we're naming our YouTube videos what do you think we should do I think you should do this this and this and you go out and test it and you come back and you're like hey inhale your recommendations kind of suck it's like all right well don't listen to me anymore but you need to have that level as opposed to being hey Neil has a really good background so I'm just going to trust everything that he does right away and even if I have really good recommendations let's just say for a moment that I'm advising I'm so popular that I'm advising 500 people many are all competitors and I'm giving them all great advice well now the the status quo has been risen and so now you're not going to see any benefits so again your question is how do I run a little bit faster than that bear how do I run faster than the other campers which is to say I need something better and so what you're what I'm seeing here is I'm saying look intuition can work well it can Inspire machine learning and AI can help scale that can help improve it can learn how you behave so the interesting algorithm for me wouldn't be answer Chris if you're really good at naming YouTube videos I would take your YouTube naming and I would build a machine learning model out of that because that data set seems to be performing better than whatever this other program's using this is is um I love your your kind of Silicon Valley nerdy humor if machine learning Bots are running off the cliff would you hey I I I've seen you see you see some of the most random like this is actually the book uh that I wrote originally started with me just telling stories of failures because I don't think there's enough of them but I just remember one time I was meeting um meeting this company one of those they didn't turn into a big one but you know I kind of have those direct to Consumer mattress companies to send you mattresses in a box and and I sat down with them and I'm like all right let me understand your business model here I was like all right mattresses you stuff them in a box you drop ship them and I'm like so you're a mattress company and they're like no no no no they're like we're a sleep technology company what does that mean and I'm like oh well we we we have a we have a a mobile app that connects to your map I'm like I don't need this I need a comfortable bed but if you want to make money if you want to get investors you need to have that big Vision that that comes with the valley um there's another one time there was a payroll company right there the digitized payroll for small businesses and I assume that like what's your business model like we are the connection between people and possibilities in their life like what does that mean and they're like well we we are the enabler of people getting access to Capital therefore allowing them to come no I'm like you write checks to people and you pay taxes like let's just really simplify the concept and the same thing though happens uh you know carte blanche with machine learning where it's like look at all these things we can do and I don't say it because I want to tear machine learning down I just worry that we enter into an era where people make bad decisions because they see what other companies are doing to say I need to position my company in this way I need to use machine learning in this way I need to name my videos in this way and they lose that ability I don't call it intuition as much as I call it critical thinking to challenge and to say this data is not all-encompassing there are other questions there's ways you need to break Beyond and that's not a bad thing that's not saying you're not data driven it's just saying you're taking in multiple perspectives from different people and you're Building Systems to objectively judge who you should listen to okay my older brother his name is Arthur if he were listening to this he's going to start scratching his skin right now because I recently having a conversation with them and he said you know companies in the future who employ machine learning will have an unfair competitive advantage and he points out Amazon and Netflix as two who are using machine learning and so he told me think about how you can use machine learning and you were saying if you want a Silicon Valley investor just mention AI or machine learning in your business model and you'll get some money so I need to rethink how I'm talking about straight so machine learning can be helpful and I love the way that you describe it where if I have an instinct on how to write titles for videos that perform well we should build a model around the decisions that I'm making what I'm looking at so that it can scale that process and I love that explanation there is a time and place for machine learning and you you have some skepticism towards it can you point to us another use case where machine learning can actually very be very beneficial for a business I would actually I would actually agree with you brother I think that if you have a critical lens towards it and you're able to objectively look at it to say look you know Google as a company uses a lot of machine learning and automation today these are the keywords you should advertise on this is a target this is how you should set the bids they're using vast amounts of data incredible Engineers to build those recommendations and I've looked at the underlying performance data it works really well and therefore I have a lot of trust for those models is self-driving cars I think are getting there even even basic things recommendation engines being able to go through millions of products and make recommendations now I know Netflix at a time I don't know if this is still true was using machine learning to recommend hit show shows I I don't know if they're still doing this um I I end up you know I end up watching dating shows sometimes on Netflix with my wife and I'm not sure I like look at that I'm like what algorithm pumped this out that said all right we're gonna get that dude from 98 Degrees and a bunch of unhappy couples and we're gonna smash them together in a house and I'm like how'd you get this like what data set brought this in and what I say is I say that the what we should look for with it and I think this is where the agreement is is that because someone's using machine learning AI it shouldn't suddenly be a gold stamp of approval to say this technology is so awesome and so Advanced that it's perfect and that we should use it and that it's going to outperform our existing techniques we have to validate it and that's kind of where I look at it when I see companies what they'll do and this is often the pitch and this is where I draw lines they'll say we're using machine learning I say can you explain it to me no it's proprietary it's black box well how can I trust the results of it well you just have to can I help can I test it can I evaluate it if I don't have those criteria in place and it's simply no come jump off the cliff with us trust me this this parachute will work then I challenge it and I think what you've seen here in this this kind of scenario that you have is to say look I tried a product machine learning had a lot of potential that's why I looked at this tool to see what it would recommend and my results have been the opposite that it's not working at least for my business and the question is to say because I believe that this technology has potential how would I build it what would I use differently what would I do to change it and that's kind of the nice part is that we are far from this being an established and mature area so these hypotheses very much could play out you could end up building Chris this could be your idea you'll come back in like two years and be like I sold it for a billion dollars a better YouTube video renamer and I Challenge almost all companies to that I say don't trust this technology blindly but have faith that it will work it will play a large role in what we're doing just make sure you're critical of it because that will be what drives the advancement more than just blind investment and blind hope I want to be uh clear here in case I'm offending anybody potential sponsors for my team whose job is mostly to edit the videos and to do some Motion Graphics to make the videos as engaging as possible they have limited amounts of data because what they do is they'll cut the video the upload it they'll call it something whatever the algorithm tells it to call it the algorithm is going to outperform their first or second attempt at titling it it's because they have fairly finite data points to look at whereas I'm on the opposite end and I'll use my data intelligence I don't know what you call it in the industry but I'm watching videos all the time from other people myself and talking to people and seeing because they literally tell us in the comments you should call the video this and if you read enough of those you start to see a pattern because they literally tell you that well we'll call that unstructured data to make everybody happy okay they have experiences I have comments it's like I didn't put it in a table it's just unstructured it's still valid so basically it can outperform depending on your data set and how how much interest and attention you're paying to a particular problem but if you're if you're using a lot of unstructured data and you're able to intuitively shape it then it might you might over perform what the machine can tell you to do so I think it's still it depends but this leads me to a different question I think it's human nature to resist change that we we want data as long as it supports The Narrative of doing what it is that we do and so we're all walking with degrees of bias some more or less and so when the machine or the data or the data analyst tells us we should do X versus Y and that's a different thing than what we wanted to do the first thing that we're going to do is we're going to try to poke holes at the data and that it doesn't work and we're going to do that and what we probably should do is just stop for a second we should have some skepticism and we should try to act on the data and the recommendations and if we're kind of a little reluctant how long do we run a test before we know that it's working or not working because I think the length of time and our commitment to it also determine whether or not it can be successful or not so that we don't just revert back to previous patterns of behavior and decision making you know when we talk about experiment at least let's talk about broadly design and that can include the length the amount of tests that we run our sample size and so on what I generally look at is I try to look at the entire options for for it so you know a lot of the tests we run we will run geobase tests which will take six to eight months for us to run you know I had a result one time from a large retailer and I brought it over to some of the professors I work with over at Penn and I said hey what do you think about I was proud of it this was a difficult test and we had to run this and I was like what do you think about I was like could this integrate into a paper somehow in research and they kind of laughed at me and they're like we really wouldn't accept that test I was like what do you mean to me it took us months to do this another generally we need to run a dozen or so tests three or four years worth of work we need to be able to show it has repeatability I said well that's great I was like but I could not tell any company that they have to wait three to four years before I can find an answer but what it avails is to say there are different types of methodologies based on the burden of proof that you're putting on somebody to validate something so in this case you could say look if you want a complete perfect answer it will take three to four years and the costs are three to four years of experimentation plus waiting for this to work in other cases there are easier methodologies to say you could do something in six to eight months you could do a pre-post test for one to two and what I really have to discussions within the organizations I say culturally I say how much proof do we need before we make a decision in which case you have to consider how big of a decision are we making can we reverse course if we need to and how much evidence do we need to see as a group that something works and I think that last one is key I think oftentimes your disagreements arise in companies is they don't have firm criteria to say how they're going to decide whether something's working experiments are rarely black and white there's always some gray area something's going to bring to be like hey this happened in the market during our test or maybe this test wasn't large enough or you know the result has some error on both sides and so what I generally say is for everybody around the table I simply say let's get agreement here what is the minimum amount of data we need until we're comfortable with the decision we're going to take knowing that that decision will carry risk and so some people in the organization say look risk is going to happen all the time and it's just part of that process in other cases they say look this is a million dollar thing we need to be certain about what we're doing and they make different decisions and there's no right or wrong answer to it it's just you'll be surprised as to how better companies are able to make decisions when before that test is run they decide what's that burden of proof how are we going to establish as an organization and what are we going to do based on the results of that test because if they can't get past that point it doesn't matter running a test if you say well we're just going to try this out and nobody agrees to say well if this test shows us this we're going to buy this product no one does it they just hope well we'll look at the data and then we'll kind of make a decision as to how we feel and that I think is the worst case because then you're just going entirely based on intuition I really love the way that you frame questions I guess you know a thing or two about this I mean just starting out beforehand getting some kind of consensus agreement and and so that when we see the data we can then commit to it because it's fulfilled the requirements that we set for ourselves it's like you're kind of Designing the the perimeter of like how far we can explore the edges the boundaries and then that way we we can look for the data and come back and say we don't have sufficient data yet but once we have that then we need to be brave enough to I think as entrepreneurs to act on it and give it time to see if it works or not that's all it is that's all but you know the reason I bring it up is because I think that I put this delicately I would say that a lot of companies don't establish that criteria and because of it they may put irrational constraints on what they're trying to do and oftentimes it's getting people out to say look how are we making a decision on this data a lot of companies confront the idea they don't know they don't have any criteria they don't have any consensus and that sometimes is a more valuable problem to tackle than whatever the experiment is focusing on is to say we have no criteria in an organization for how we make a decision so why are we running all these tests and then by doing it in advance where it's a non-discover problem to say look how much evidence do we need before we take this action sometimes we work in the academic research space where I can give you 60 000 pages that says this is the right thing to do for your business and companies say well we're going to test it first I'm gonna say so you need 60 000 and one pages of data that's the threshold it's like oh you're right that doesn't make a lot of sense but it's until you objectively get out there and say this is a criteria you need this amount of evidence to do something they don't realize how much they're slowing down their organization their decision making I like the way you frame that question to make them rethink their needs for more testing well done thanks there's two more questions I need to ask you before we run out of time here the first question is and and it's about questions because you talk about the in the in the in the book like you're going to learn how to ask the right questions that we can anticipate our customers needs and you you seem to be very good at asking the right questions is there anything else that you can shed light on in terms of how we can discover what are the needs of our customers are I I go to uh to a slightly deeper level the impetus behind that section was to say that oftentimes people look at data collections what do we have available do we have this metric do we have this data removing themselves from the possibility that they can ask customers and that process seems to have the most friction in a lot of organizations especially retos we want to ask our customers more things how did you get to our site what do you think the difficulty of companies doing that is just incredible oh we don't want to add more friction to the checkout process or this is our annual survey and what happens is that Curiosity dies or if nothing it's marginally replaced by saying well do we have a data set that says how they found us well we have some rough referral data in our no ask more questions be curious rotate those questions it doesn't have to be part of a large survey if you want to know when to ask questions ask it on the thank you page turns out the height of trust with most consumers is after they give you money ask them questions instead just being like here's your order number rotate those questions and learn about it be curious and that's really what that chapter is about is to say if you have a conversation with someone and Chris you and I are sitting here and you're just talking and I don't ask you a single thing I'm just listening that's not engaging because I'm sitting there I'm like no I what what do you think explain that a little bit more have that conversation with your customers the only thing that I put on it is I say as we do with experiments know what you're going to do with that answer so just go what do you think of my ads what do you think of my products no know how those answers are going to change your future Behavior yeah oh do you want to do you want to go skiing sometimes why are we going skiing no I was just asking that's a ridiculous question we have a question how much are you spending with other competitors you know what was a feature that really brought you here how long did you think about these products all those things enable that type of curiosity go forward and give you new opportunities that you might otherwise miss those are some great questions super actionable and now I have to call Ben and say Ben are we asking these questions on the thank you page because we need more data that was awesome thank you the time you do it okay the the second this is the last question here which is you you kind of mentioned this uh just slightly at the beginning of our conversation about even for social media a lot of people who listen to this podcast are very excited about trying to grow their influence and Authority on social media they have expertise but they can't reach an audience and we have limited insights from the different social platforms so if I'm trying to grow my presence and my authority on say something like Instagram how can I use data to allow me to understand what's working what's not working so that I can make content that's going to be more attractive to my my audience uh here here's kind of the way that I approach it social media is attractive because it gives us rules for conversation and it gives us feedback on it if you and I have a conversation or if I message to a thousand people say to conference and there's no feedback mechanism maybe I can't see them maybe I can't hear from there's no survey I don't know how I did and social media overcomes that by a rating your conversation if you do well they promote it you get more likes people are saying do you comply with the rules of the community and so when I look at success of social media from that lens I say really what it is is to discover to say how closely can you conform to the rules and best practices of that particular group what's successful on Twitter will not be successful necessarily on LinkedIn when people are looking at viral content that's like ah something that conform to all the rules of what the community expects and so from that lens when people are saying well how do I become more successful successful on social media it's really saying well how closely can you comply with the rules that they put out almost prioritizing them over your own Brands and Ambitions like the way you present yourself the way you present your products the way you write copy all have to comply with it and I don't really think that that's an interesting problem to solve because when I look at it I say well say you have you know a million people following you and they you see these these articles come up every now and then where you have these influencers with all these followers and then nobody takes action on what you said and I think there was like this one famous musician I won't name names that had something like you know 20 30 million followers and then wrote a book and I think you know only about fifteen thousand of their followers actually bought a copy of the book despite all their promotions and I say is that really worthwhile and let's tie back to the kind of the larger theme we're talking about we see how all this stuff is connected if you have a CRM database of a million emails and you send out an email and nobody replies and nobody buys your product how do you feel do you feel great because you sent out a million emails or do you feel kind of terrible because you have a million emails and nobody cares about the message you're sending and so I go back to the question I say oftentimes social media rewards people with quantity more than quality and I think that it's a rough proxy and I think we get distracted by but my challenge would be is I look at the success on social media as I do with any other channel if you put out a message if you put out a request do people respond to that more than just arbitrary buttons to say they're they're engaging with it what a lot of people find and this is a discouraging part about social media is that they find out when they do put out these requests to say hey I have 50 000 followers I say buy this thing listen to this thing and they get such a low conversion rate over it and I say and I was like well that's kind of I look at all these people and nobody's interacting and nobody actually bought the thing I put out there and it says that's fine but I say what that does is is that reveals that there's a different set of rules there's one set of rules for volume and activity there's another set of rules for good relationships and those rules are a little bit harder to discover but I say that's the that's the game you actually want to play so if I'm in the influencer game I don't care how many people follow me I care how many people act and so that's why I prioritize I say what you want to do is learn about as many people who take action who click on something who go to a website ask them questions about why they did and then figure out what content relates to them I'll give you one quick example on it there's a case study of Electronic Arts a video game company where they put out 16 different display ads for promoting a mobile game and the one that brought the most downloads the most volume was the one that was least attractive to the people that would actually spend money in the game and when you think about mobile gaming 98 of Revenue will come from 0.2 percent of the customer base and so for them the success of their work came down to customers and barely register if they tried to look at what was impactful for downloads and I say the same with social medias if you look at say what's impactful for likes or retweets or anything whatever metric you're looking at it's likely going to be different from the people that are going to support your Enterprise that are going to spend buy and want to be connected and relatable to your brand that's the end of my TED Talk there social media I'm I'm I was nervous listening to that like oh my God because I'm at like 800 000 followers on Instagram highly engaged audience I don't know you have the rules and and the thing is look if you have 800 000 followers and you put out a message and say I have a new product and they buy your product and there's that exchange of value then I say you found the Best of Both Worlds right but I think all too often people would say look it's kind of the assume this idea the if you build it they will come mentality it's a I'm going to capture a whole bunch of customers and then they will see how amazing I am and they will love me uh daily deal space right where like restaurants are like I'm gonna give away my products at 80 off and I'm gonna have all these customers come in and give me their email addresses and then none of them come back and so I always look at that connection to say it's not simply the volume and how many people raise their hand it's how many people are willing to follow you in the next step and that I look at a success and those rules are harder to identify right because you need somebody to take action buy something as opposed to clicking a little button to share but I think that they're all that more important for anybody that's pursuing that channel that could be the beginning of a whole nother conversation that you and I can have but as it is we're out of time my guest today has been Neil Hoyne he's written the book converted this is a book that if you've enjoyed this conversation as much as I have you definitely need to go out and pick up this book Neil thanks very much for jumping on podcast today hey Chris my pleasure thank you for having me hey hey my name is Neil Hoyne and you're listening to the Future