Matt Feeney: [00:00:00] Hello everyone, and welcome. My name is Matt Feeney, I’m a senior research manager with Medallia, and today I’m gonna be walking you through our session uncover what customers want, how insights, impact, relationships. So what does that mean exactly right. What are we gonna be talking about? As you’ve no doubt noticed over the past few days, one of the central themes of experience this year is gathering information from various inputs to better fuel your decision making.
This is Medallia, right? So customer experience remains at the center of everything we do. However, today we’re gonna be zeroing in on a slightly different signal but is related somewhat specifically market research. We’re gonna be talking about why it’s important. We’re gonna go through different questions that can answer and how we can get at those answers.
And also we’re gonna be providing some real business examples. Of how these learnings are impacting businesses and allowing them to both minimize risk, identify opportunity, and just overall grow closer to their clients. So as you see, we have a [00:01:00] nice agenda here. Part of this is going to have a guest speaker, which I’ll introduce shortly.
But in the meantime, wanted to talk through market research. I’ve been in the industry for, oh. Two decades. Now, that sounds really long when I say it out loud, but basically the best part of my job is being able to see clients find that aha moment, right? The thing that can actually solve a direct business problem and prove itself.
And that’s very often accomplished through market research. There’s just so many tools you can use within that to really get it what you need. I’ve seen bus businesses pivot. I’ve seen them evolve. I’ve seen them even like reinvent themselves based on research findings. Thought it’d be interesting to start off with some examples of this.
Starting on the top left, we see telecom provider, so I’ve seen a major telecom provider that was dealing with increased expenses related to their customer service costs. They didn’t know why. So they decided to actually employ some [00:02:00] qualitative market research. This was done through the use of focus groups pretty easily.
They were able to, at that point, identify that the root cause of all these expenses was the fact that their bills had become overly complicated. If anyone has a telecom bill, you probably know what I mean, line after line, they become tough to look at and just causing a lot of confusion for their clients, which was resulting in all these calls, all these emails.
So from there, what we did is, was we pivoted over to some quantitative research. We put together some surveys that allowed them drill into that. And these included heat map analysis, which let customers actually view a bill online and click on the things they liked and didn’t like. And then tell us in their own words why.
This was super powerful and it resulted in what was essentially a roadmap on how to build a better bill for them. Takes a little bit of time and they’re creative, but after many months, they actually rolled out an entirely new bill. Probably the only bill I was happy to get right, because I’m looking at this and I’m seeing all these changes that were directly [00:03:00] tied to feedback from their customers.
It was awesome, and they actually told us about a year later that. Their customer service expenses had peaked right around the time they made this switch and started going down incrementally over time. So it was a big win. All fueled by market research, working with a major television network. We used to do pilot testing where we would basically send out surveys to people around the country and they would actually watch these different programs that networks were considering putting on air.
Based on the survey data, you basically will hold up all the results against normative scores to see what’s performing and what’s not performing well. One, that one that pops in my mind is to show, you would all probably know the name of it was strange because we were getting conflicting results, which was rare.
They were scoring well on pretty much everything, but their intent to view scores were plummeting. It didn’t really make sense. But when we started diving into the analysis, what we found was there was a high correlation in the in negative scoring for one character. And [00:04:00] those that were not likely to view the program, we called this out in analysis, made it really clear.
And, fast forward a few months, they actually selected that program to make it on the air. But when I turned it on, it had an entirely new actress in that role, new character. They killed the character and they rewrote it, re-shot the pilot and ended up going with it. That show ended up being a mainstay on primetime for roughly seven years.
I think it was. It garnered many Emmy nominations, including a couple nominations for the actress that was brought in to replace the poorly performing character. So again, research led to a better product. National beverage chain, this is a more recent one. We’ve been getting a lot of work surrounding pricing, right?
Turbulence in the economy. It makes sense. So for this national beverage chain, they understood that they had a very strong brand for their clients, right? They were very well respected, regarded, but what they didn’t know was how their prices were perceived. They had no idea how they were viewed in relation to [00:05:00] competitors in this regard.
So using one of the many price tools that you can use throughout market research, and we’ll look at those in a minute, we were able to basically rightsize their pricing and show them that they were in fact. Underpriced in the view of their consumers and a lot of their items had room for growth. Fast forward, they updated their menu boards and are now making higher revenues on those items, and they were able to do it with confidence knowing that their customers weren’t gonna turn on them.
That’s research and action. Bring it back to customer service though, or customer experience rather. I went through a couple iterations on how to show the relationship between these two. At one point there was a Venn diagram. I think there was one of those circular charts with all the arrows that you see on all the presentations, right?
But this is what I latched onto and I couldn’t get outta my head. A good way to view the relationship here. ’cause there is a lot of confusion is customer experience is really, it’s what you can see, right? It’s done at the touchpoint level, it’s immediate. It lets you act [00:06:00] tactically to retain customers and make sure they’re having positive experiences.
Market research, however, is extremely useful because you can ask anything and you can learn so much, right? It allows you to act more strategically to solve problems and get in front of them before they actually happen. Here’s a good example, right? So what do we learn from cx? You can get things like satisfaction, immediate positive and negative feedback.
You can get recommendation scores, impact on future behavior, impact on brand perception. You can get it at a high level. At the transaction level, right at that touch point, right? But look at the gaps that market research can fill. You can find out at a broader level, how is my brand viewed? How are prices perceived?
What’s drives my customer’s behavior and who are my customers? That gets overlooked by so many people, but it’s extremely impactful when you apply market research and get to learn that. So I thought what would be a good idea is to take each of these questions and go through and show them how we can answer them through [00:07:00] market research and show you a quick example of output.
This first question, how is my brand viewed? This is one of the most common things we do in market research, and you can get it in a number of ways. You can look at brand associations with different characteristics, attribute perceptions, brand KPIs, the funnels that you’ve always seen, and you can trend them against averages and competitors.
Attribute perceptions is one of the most effective, I feel. This allows you to easily ask at the survey level, how important are each of the following. And then how closely do you associate your brand or competitive brands with those features? With that yield you is actionable insights. If you look in the top right quadrant here, you can see things that are rated as of very high importance to customers.
Whether you perform very well, price, customer service, that’s great, but if you look to the top left, things that are very important that you’re not necessarily performing well on. In this case, order accuracy. [00:08:00] This allows you to prioritize changes in your organization and impact your relationship with your customers directly.
Let’s move down to how are prices perceived. The price question is amazing because there are so many different research methodologies you can employ to really get at that and drill down things like demographic differences that you can call out. Gaber Grainger, which is a statistical methodology that basically tests you at different price points to see likelihood to purchase.
Van Westendorp, which is more of an open end, and we’ll drill down to that one in a second. And then value base, which is really just say, assigning different values based on different characteristics. Looking into Van Westendorp specifically, you really just ask, looking at this item, at which point would you consider it to be too expensive, open-end, expensive, but acceptable?
Again, open-end, a good price that I consider a bargain. You’ll get the actual amount there or cheap to the point where there’ll be quality concerns. This [00:09:00] yields extremely impactful data. You can find the average for each of these quadrants, and then you can find out where your specific product sits within each.
This shows you how much range you have to grow or if you’re overpriced, what you need to do to scale back what drives purchase behavior. This is huge, and we’ll talk a little bit about these in a case study in a moment. But we’ll talk about benefit importance. That’s one way to get at that turf analysis, which shows you what combinations of features and benefits will get you the most satisfaction from the largest base of your clients Offer rankings, which is super simple to apply where you’re just looking at what’s most important or a key driver analysis, which we’ll drill into in a second.
This can be calculated by basically looking into how satisfied are you with these different features. How satisfied are you with the brand overall? A few more questions and you yield basic results that tell you what impact [00:10:00] each of these elements has on your satisfaction and your likelihood to purchase.
Gives you immediate direction through market research. Lastly, we’ll talk about who are my customers, right? That kind of like broad question. There’s a number of ways to get at that, right? So pain point analysis, if you’re looking to manage problems. Who are the people that are dealing with specific problems and how can I fix it?
For those groups, customer journey, you can look at the different touch points and figure out where they’re meeting resistance or where they need to improve customer segments where you can just basically take a number of different variables and create unique segments for the customers that you can then activate against, or buyer personas that you can actually build out with some of your more common customers to account for most of your market that you can then market to those.
Buyer personas, for example, this can be basically obtained very easily through asking what brand do you purchase most often, and then cross-referencing against an any number of different demographics or firmographics. [00:11:00] The results are buyer persona output, where you can now see what percentage of your market is occupied by each of these groups, and how to more effectively market to them based on their behavior.
Again, market research in action. So with that, I am going to call Keith Romer to the stage. He’s the director of research and insights for realtor.com. He is currently doing a great amount of work basically marrying market research with CX data to get stronger insights for his clients. Hi everyone.
Good to have you, bud.
Keith Romer: Yeah. Thanks for having me.
Matt Feeney: You wanna tell everyone a little bit about yourself?
Keith Romer: Sure, yeah. Yeah. As Matt alluded to, I, so I lead research and insights at at realtor.com. Been doing this for about three years here. Prior to this I was at I’m really excited about this talk track.
‘Cause I think that, we are talking a little bit about merging. CX and market research together, and in previous organizations prior to [00:12:00] realtor.com, those two things typically lived in separate verticals, under different leadership. When I was at Petco, there was just a lot of crossover that would happen and it just felt like there was a lot of barriers and walls to try and, get synergies between the data. At realtor.com I, brought all that under one roof underneath myself. So my team does all the market research for both B2B and B2C, but then also does all of our CX work as well. And we’re able to really drive a lot of triangulation, connecting a lot of dots, and making it a lot more actionable for the for the organization, which I’m really excited to, to dive into a little bit later.
Matt Feeney: Awesome. So we thought we could start off with a few questions. The first one being, what are some of the things you’ve gotten from market research that you couldn’t necessarily get elsewhere?
Keith Romer: Yeah, I think I have a really good example from my Petco days. I. Back, Petco, a retailer.
One of the things that we were looking to get into was was services a little bit more. We wanted to up our grooming, we wanted to get into vet a little bit more. And one of the executives at Petco had said, for the grooming salons, if you go into a grooming salon, if you guys have any pets or whatever, it’s loud.
The dogs [00:13:00] are there’s this, it’s chaotic. And so the executives had like. Put that as, oh, the dogs are scared. There’s this fear. And they wanted to do this like fear free experience for the dogs, for the pets. They wanted to do this whole marketing campaign around it. And I just had sat there at this, in this executive boardroom that where this meeting had or this idea had come up.
And I was just like, we probably need to pump the brakes on that a little bit. I’m a parent myself. I don’t know if I’ve ever truly seen my dog like fearful. Scared. It is a very aggressive emotion. And I had suggested that we do some re some market research with pet parents, dogs specifically just to understand these emotions that themselves, their perceived emotions that their dogs were were experiencing as they go into a grooming salon.
And what we ended up, long story short, we ended up learning in these interviews with pet parents, is that it was actually anxiety. Anxiety was like the fear that was more manifesting and it was actually manifesting more through the pet parent than it was the pet. The pet parent was actually the one that was experiencing the anxiety more than the pet was.
But there were realities where there was anxiety that was [00:14:00] happening. And from a marketing perspective, that’s huge. Saying Hey, your dog’s not gonna be in fear here. Raises a lot of alarm bells for pets. Should,
Matt Feeney: why should they be in fear? Why should they be?
Keith Romer: Yeah. Yeah, exactly.
But when you start saying anxiety, it’s a, it just, it resonates more with consumer. So we were able to get more consumer friendly language, even in the products that we were offering. We learned that lavender was was like a calming scent. So it was just like anxiety free. Package that we offered, we would’ve never been able to learn that if we hadn’t gone and done the research with consumers to, to understand that.
Matt Feeney: That’s interesting. So taking that a step further, and it dovetails nicely into question two. Can you given a time where research findings changed the company strategy, obviously in that you were able to, ID a specific word that you just realized you’d have to pivot on Totally.
But are there any other examples you can give?
Keith Romer: Yeah, absolutely. Like in that example definitely was like a change in what we had planned on from the strategy. But even@realtor.com, we recently, Matt had alluded to the work of a turf analysis or a max stiff and a turf analysis.
And so we had [00:15:00] actually worked with Matt. On that to understand the features on our site realty.com. Historically, the past few years that I’ve been there, it’s just a, it’s a feature game. We’re in tech and it’s basically let’s just pump out feature after feature.
’cause we just think that’s what consumers want. And so we needed to level set and just understand that we were doing marketing messaging and all kinds of stuff where we were just throwing out like like the, we were launching ads where it was like, oh look, you can draw a circle on map with your finger and whatnot.
And like me as a consumer, I’m just like, whoopy, do you know, like I trying to buy a house here? Yeah. And we just, we didn’t know like really what the priority. Of like features were on our site. And so we ran that max stiff. And the max stiff is essentially a prioritizing exercise.
What it does is it puts features up against themselves time after time Again, instead of doing like a rank order, this gives you like a composite score for each one to say, where does everyone kind of rank rank within each other. Anyhow, four basically rose to the top.
Everything else below that [00:16:00] came down. And what we pretty much learned was it was like, we really need to fix. And focus on these four core features, and especially in marketing, only focus on them because that’s what people are coming to our site for. And so we were really able to change the company strategy of let’s just hum out all of these features to.
We need to work on these four features because frankly, we were also running CSAT s on the site as well, and we weren’t getting great feedback on how those were even operating in the first place. So it was, we need to make table stakes. Let’s make sure our table stakes offerings, the ones that matter to consumers the most are performing at best.
And then we can start working on feature after feature instead of just having a handful of things that just work very, poorly, essentially.
Matt Feeney: That makes sense. So without that you probably would’ve been spending a lot of money on things that don’t necessarily move the needle.
Keith Romer: Absolutely. Yeah. Yeah, for sure. And even now, if you go on our site or our app, there’s a bunch of just random features that like, they don’t move the needle, right? But it’s Zillow launched this, so we gotta do, go do the exact same thing. The reality is like we even know, again, from market research.
The [00:17:00] consumer that goes to Zillow and the consumer that goes to us is entirely different. A lot of people that go to Zillow are just browsing. We call ’em looky-loos. They’re just looking around. Neighbor’s house went up on the market, they’re nosy. They want to know oh, how much does that cost? And so that’s what a lot of Zillow consumers are, but we actually realize that a lot of our consumers are more, they have higher intent to buy. They’re more engaged in the process. And so we need to have features that are like those features that they want as they’re like in that part of their journey.
Need to make sure that they’re up to snuff. And so that, that was a big finding for us.
Matt Feeney: Related to that, you’re talking about your customers. You’re in a unique position where you have multiple customer types that you’re trying to really serve. Can you talk a little bit about that, how it varies with research?
I.
Keith Romer: Absolutely. Yeah. So we’re a double-sided marketplace. Consumers like me and you, we go into the, to the app, to the site, whatever, and we’re searching for a home in whatever neighborhood we’re trying to look. So those are our consumers, but our customers are agents and brokers. And they’re actually how, believe it or not, how we make money.
So they’re the ones that are paying us for when you see [00:18:00] that little button on the, or when you see that little form on the side of the of a homes page where it’s ask an agent about this property. Agents and brokers pay us for a certain number of those basically per month. And that’s which I
Matt Feeney: had no idea about before.
Keith Romer: Yeah. So I
Matt Feeney: started working with,
Keith Romer: hopefully I’m not giving away a bunch of realtor tr secret secrets right now, but I don’t think I am. But anyhow, they’re incredibly important to us because they’re the ones that actually pay our bills, but they’re interdependent on one another without. The consumer the customer doesn’t have any leads.
And so we don’t really make any money off the consumer, but they’re a huge focus Sure. Of us because we have to have them in order to have the folks who are paying the bills.
Matt Feeney: Makes sense. So going back to research in general, like how has your view of research changed over time?
Keith Romer: Yeah. This is an interesting one.
I started my career off on the agency side, so working in a market research vendor. And, as I was being brought up and as you should on the vendor side, everything is so pure. Methodology has to be, so tight. And the, just the purity [00:19:00] of the research was so important. And if somebody’s gonna pay you, thousands of dollars to commission a study, you better do it that way. But as I transitioned over to the client side, I started to realize. You don’t have time for the timelines and the perfect methodology and all that kind of stuff, and you do have to make concessions.
And that was a huge shock to me when I first mo moved over. ’cause I was like, no, what about the purity of the research? And so I had to make really hard pivots like really early on. When I switched over to the client side. To be able to be as methodologically pure as possible, but then scrappy and in other ways to cut down timelines and to maybe the way that we recruit people is a little scrappier and whatnot.
But the core methodology stays, as pure as possible. But even then, sometimes you do have to. You do gotta bend the rules a little bit.
Matt Feeney: You don’t skew too far from that. I’m a research purist and I work with you pretty closely and
Keith Romer: yeah,
Matt Feeney: You definitely make concessions, I can say, but they’re ones that you could totally validate
Keith Romer: a Absolutely.
I think at the end of the day, like I don’t wanna put myself out of a job or whatever, but [00:20:00] as long as you’re talking to your consumer or you’re talking to your customer. Like you’re 90% of the way there.
Matt Feeney: Yeah,
Keith Romer: Just like asking questions and being with them and whatnot. So I think that’s that.
But at the end of the, when you get into quant and stuff like that, like the surveys, that’s where you gotta be really pure.
Matt Feeney: Yeah. I’m sure everyone here has heard about it. AI in the past couple days.
Keith Romer: I’m gonna, I’m just gonna be fully transparent. I have absolutely no idea how AI fits into market research.
And what I mean by that is you’re not
Matt Feeney: alone.
Keith Romer: Yeah. Yeah. Like on the qualitative side with market research, it’s so deeply like empathetic and human natured. And what I mean by that is when you sit across a table from someone, it’s just something that, you know, like developing that relationship, making a consumer feel comfortable so that they can, open up to you and be transparent.
It’s just something that I don’t necessarily see a bot having. But then on the other hand, there’s also the observer piece of qualitative research that I just never see a machine being able to replace. And an example of that is when I was on the agency side I used to do a lot of work for Hershey and they had just wanted, they were [00:21:00] getting into the club space.
They were really leaning heavy into Costco and Sam’s and BJ’s. And they had this Twizzler pack that they were getting ready to launch new packaging and whatnot, and it was this big canister with a lid. And so we had people doing in-home usage tests with them. And this guy, I remember this so vividly, it was like one of my first couple years of working in market research, but this guy was like, man, I really love this container.
I love this container. Oh, I love the I love the design of it. Oh, I love Twizzler so much. But, and he’s doing this, he can’t open the damn lid. And so like without the kind of an important part. Exactly. Without the observing part of that, we would’ve never unlocked the fact that there’s like actual issues with the lid mechanism and whatnot.
Whereas if you just listened to the transcripts or if you were just, like going on what he said, you would’ve said, oh yeah, this is fantastic. Zero pain, great piece of packaging. This let’s launch. And so I don’t know how AI fits in. I was talking to Matt a little bit earlier about this, in market research, there are repeat and it’s a lot like CSAT or CSAT surveys or NPS surveys and stuff like that where it’s repeatable. Like once it [00:22:00] gets programmed, it’s there. When market research, there are things like brand trackers and whatnot that are also just set it and forget it kind of thing.
I could see AI being able to just basically take over once the human builds. And then for repetition. Outside of analysis of like raw data and stuff like that, I just, I’m really struggling with how AI fits in and it’s something that I’ve tasked my team to try and learn more about because, to not be a fear monger if we don’t learn how to adapt and bring it in to our team and be masters of how to leverage it as supplemental.
I have concerns of some executives saying, oh, I found this tool, it’s gonna replace our market research team without actually knowing that it can actually do that. And like I said, I think that’s a far stretch or whatever, but I think that we as a, as an industry, need to embrace it. Yeah. And learn how to leverage it as supplement and not a replacement.
Matt Feeney: I think that’s fair. And I think it’s gonna be iterative too, right? Because you’re gonna be able to see where these gaps are, where you need to fill it in with research versus AI or vice versa.
Keith Romer: Absolutely.
Matt Feeney: Yeah, absolutely. Cool. So I know you prepared a [00:23:00] case study, you could walk everyone through that, give us an example of some of the work you’re doing in market research.
Keith Romer: I did, yeah. Yeah. You wanna
Matt Feeney: go with that?
Keith Romer: Yeah, I’ll take this. Alright. Probably stand for this one. Why not? Okay. So like I said realtor.com double-sided marketplace. So we have our consumers, we have our customers, and we had a leadership change year and a half ago probably the previous CEO. Big CSAT guy.
That was our main KPI for tracking. How’s the company doing? Both on the consumer and the customer side? We had a new CEO come in and part of his perspective was CSAT is a little bit too tactical, especially on the B2B side with our customers. And so he wanted to switch our main KPI over to NPS.
So NPS being a much more relational metric, wanting to understand how are we doing, as as a company that. We rely on those relationships with these massive brokerages that are spending millions of dollars a years with with us for the, for these leads. So anyhow, we made this switch.
We had been learning in csat some of the things that NPS brought to light and our NPS score was [00:24:00] alarmingly low. And it really highlighted some of the things that. Came up in csat, but we didn’t really think we’re like all that big of a deal. It was things like poor lead quality, poor lead volume concerns with ROI in the, in our csat.
So we were intercepting people as they were in the experience. Very tactical. How is this work? How is this working? Are you guys satisfied with the experience? And some people were saying, oh yeah, the lead quality that you guys have sucks and all that kind of stuff. And we’re like, okay, cool.
Like it’s just a disgruntled person. But we didn’t realize at what volume. That was actually true with our customers. So anyhow, so the NPS survey really highlighted just how big of a deal, lead quality, lead volume, and ultimately ROI that those two things ladder up into ROI really were for our customers.
And so macro conditions play a role in this, right? The housing market sucks. There’s low, there’s low inventory out there. House prices are crazy. We have mortgage rates through the roof, so we are dealing with macro conditions and so [00:25:00] things like lead volume are somewhat out of our control.
We can’t make people go buy homes, right? So there is some of that’s out of our control, but the lead quality is stuff that we can have in our control. So some of the things that we were learning from NPS was like, I’m getting fake people. They’re already working with an agent. All they wanted to do was just schedule a tour.
So there were just these things that were bubbling up where it’s very low hanging fruit for us. But what we really needed to do was like, it was two-pronged. We needed to understand who these lead submitters were from a consumer aspect, just to understand like, why are you submitting a lead on our site so that we can make sure that we’re serving them to serve our customers.
And so what we did. Was we did two pieces of research. We did qualitative with our customers, qualitative with our customers to understand. Okay, so when you guys say lead quality sucks, what do you mean by lead? What is a good lead to you? And you might hear me say hand raise, by the way, too. A hand raiser [00:26:00] is we call, that’s what we call them internally, like a lead, like they’ve raised their hand to be contacted by an agent.
So that’s a cute term. But so if I, if you hear that’s what I’m saying. But anyhow, so we talked with customers to really understand what does good lead quality look like? What do you mean when you say, I want a good, I want good leads, I want better leads and whatnot. We wanna understand what is the pitfalls of us versus, I’ll name the competitor Zillow.
That’s our biggest competitor in the house. Like a lot of times we hear them say, oh, Zillow this, Zillow that. And so we want to understand what’s Zillow doing better that we’re not, and how can we improve to make sure that we’re meeting those expectations? And just general, what else can we be doing better today?
I don’t have a slide on what we learned there and whatnot. But essentially what we learned was there’s a lot of stuff that Zillow’s doing to vet their leads before they even reach the brokerage. We have a concierge service that. In between the time that consumer says, Hey, I wanna be contacted, and we hand that off to a brokerage.
We have concierge service [00:27:00] that asks some questions and it’s things like, have you been pre-approved or are you working with an agent? And all that kind of stuff. We just weren’t doing a good job of doing all those vetting questions. And so what was happening is these these leads were coming in and the brokerages were asking these questions.
And they’re like, why are you giving me lead? Like they’re already working with an agent. If you’ve already asked them that you’re working with an agent, why are you sending them off to me? I can’t contractually even touch this person. So just, we just learned a lot about how we can make sure that we’re vetting leads better on our end.
And so that was a great learning there. On the consumer side. We wanted to understand who are you submitting leads across multiple places? Are we your only place? Are you going to Zillow? Are you going to homes.com? Are you going to Redfin and submitting leads on the same house? Are you doing it on multiple different houses just to understand their behavior so that we can understand a little bit better on how do we serve these consumers up to our [00:28:00] customers?
And some stuff around what drives them to submit a lead and stuff like that. And so what we learned on the consumer side was. It’s almost embarrassing. 25% of people didn’t even know that they submitted a lead. We were tied there, like we didn’t, I didn’t even know I did that. Which was fascinating to me.
And also is a problem from a concierge or like a follow up side, right? If we’re not following up with that lead and saying Hey, so if we’re not hounding them to get them to one of our customers that just seems like a little bit of an issue on our side. And so we, that, that was a huge unlock for us was that they didn’t even know that they were contacting us.
We also learned that there was a bunch, they didn’t even know what site they had done it on. So if realtor.com had reached out to them, they were, they would be like, I, I didn’t even know what site or what house I asked about. Can you remind me like what house that I was even that I was even asking about?
So that was a big a big learning for us. I had alluded to this a little bit earlier, but like scheduling a tour, people just wanted to schedule a [00:29:00] tour to go see the house. They didn’t want to work with a, an agent, they just wanted to go see the house. With the new. There was a bunch of legislation that happened with the NAR.
With the new legislation, you don’t technically have to have a buyer’s agent anymore, so there’s a lot of stuff going on. So some people are testing the waters with that. And that kind of played into there. But another big unlock for us to help sell on the customer side was that half of hand raisers said that they planned on using the agent that they did contact through realtor.com.
So for us to encourage folks on the customer side. Every lead is winnable. It’s what we preach. It’s like they’re not all winnable. We do realize that, but at the end of the day it’s not the 10% that you’re converting because 50% of people say they actually plan to work with you.
But if you’re not putting in the effort, and if you’re not gonna spend the time working on them, then that’s like a, that’s a you problem. That’s not an US problem. So all that to be said. We leveraged market research to really diagnose the why [00:30:00] behind the issues. In our experience, NPS was a mat, is our KPI on the B2B side of the business.
That is the number one thing that we focus on. Bonuses get paid out on that. So you can imagine a lot of executives focus a lot on that number. So we really wanted to make sure that we were understanding all of the why’s to help drive that number up, not just to drive the number up for the sake of driving the number up, but actually.
On diagnosing, what are the issues that we can make that number increase and serve our customers well? This all stemmed from an NPS, like a customer experience program, but it’s the actions that we were able to work on span the whole company, like marketing, right? Driving top of mind awareness and adding more.
realtor.com branding. The fact that somebody submits a lead on our site and they had no idea that they did it or they didn’t know where it was coming from. Our emails aren’t branded the right way. How we’re reaching out to people and following up. We don’t have the, we’re not stressing realtor.com enough.
There’s just across marketing, we are able to work on some things [00:31:00] there. On the product side, we need to create some points of differentiation. Consumers aren’t able to tell, who, what site they’re on, what they’re doing. We need to be able to create some points of differentiation, not so much from the let’s just feature.
But let’s just start developing some things that really start to sep separate us from everyone else because let’s think about it. If you guys were to all go onto homes.com, Redfin, Zillow and realtor.com right now, type in the exact same address or neighborhood or whatever, you’re gonna get pretty much the same stuff.
So there really isn’t a whole lot of differentiation in this space and there’s a lot of opportunity for us to do that. I think the most damning of all the things here, adding a scheduling a tour feature, so we only had that in 10% of the markets and it was working. So why we didn’t have that deployed across all 90 or all a hundred percent.
When that was a massive driver for why people were submitting a lead in the first place was a big unlock for us because then we can, on our backend, making sure that we’re segmenting those out and we’re sending those [00:32:00] people to the right type of the right type of concierge versus off to a one of our customers who’s paying for that lead, who can’t actually capitalize on it.
I believe that’s my last, that is my last slide.
Matt Feeney: Yeah, that’s it. Oh, go ahead. I was just gonna ask, does anybody have any questions in terms of any of the research methodologies or case case study example?
Keith Romer: I’ll be the first to admit, I actually don’t like MPS admittingly typically I’m pretty anti NPS because I do think that it’s a very black box metric in the sense of you’re.
I don’t know if you guys have ever seen it. It used to be a meme or a gif or whatever about, I think I said that wrong gif, but whatever. No one knows
Matt Feeney: how that’s really, it was like,
Keith Romer: it was from like, it was like from a Microsoft NPS survey and it was like, how likely are you recommend windows 10?
To, to a friend or a family member or whatever, and the response was, I need you to understand that people don’t go walking around recommending software programs to, to friends and family and whatnot. And I very much hold that belief as like there, that’s, there’s an [00:33:00] inherent flaw with NPS because of that.
It’s two questions. What’s the number and why? But I do think that for like B2B side of things, it is really helpful because it, it is a relational metric. And I do think that on the B2B side, people are making recommendations, right? So I do think that, that it’s a little bit more fitting there.
But yes, it was a top down thing. As market researchers, we also, our job is to triangulate, it’s to pull data from other studies and all that kinda stuff, triangulate to tell a story. And one of the things that we knew from another study as well is that. There’s a lot of people that don’t even know what happens when they hit that button, so they simply don’t, they simply don’t touch that button because they have no idea.
What’s gonna happen? They think that they might get bombarded by calls. There’s gonna be 15 agents calling ’em all over the place. So they just, they choose not to. So there is a whole body of work underway to understand how do we educate the consumer without cluttering that experience and making it clunky and all that kind of stuff on what actually happens when that button gets hit.
And our AB test and stuff like that, that we’ve done, we’ve seen like insane amounts of of follow through, like more people submitting [00:34:00] leads, which that’s a big metric for us is like, how many leads are we getting per month? And whatnot. So that’s like a big thing. That’s a big body of work that’s happening in place.
But for just some of the process improvements of making sure that, like that handoff that’s happening with our concierge service to the to the brokerages and impacting that, those conversations and making sure that the leads are more vetted. We’re already seeing NPS scores go up and that’s just within a quarter’s time.
So even in just a small amount of time, just having that impact over NPS has had like drastic just changes in. You can only imagine what sitting in a boardroom with the C-suite looks like. After the fifth quarter of a declined NPS score. And like it’s like a funeral.
And I tell you what, like in they shoot the messenger, right? I, it’s like it’s my fault that the NPS scores are down. What all I’m doing is just running a two question survey for you guys. But yeah, so to see those numbers go up ’cause we’re improving the customer experience and making it easier for consumers to understand what’s happening.
Has been great for morale, the [00:35:00] culture and also just for the experience overall.
Matt Feeney: Just to piggyback on that, one of the other things that you do, and we have a number of other customers that do it as well, is you’ll go off on your own. You’ll see some of these findings from your larger studies.
And I can see your instances of you actually jumping in and creating these other surveys and pushing them that are more targeted to some of those specific findings so you can activate against them more effectively.
Keith Romer: Absolutely. I will say that sometimes I get my hands slapped for going off rogue too much.
But it’s all for the, it’s all for the good of understanding the greater good. Yeah, the greater good of understanding what’s going on. But yeah, AB absolutely, just being able to identify that, like what’s. Sometimes it’ll, it might be trivial. It might just be like something that pops up like a handful of times and it’s okay, look, maybe other people aren’t mentioning this because it’s just not as important as.
The ROI and the lead quality and all that kind of stuff. And so we will dive in to understand just how prevalent is this. Yeah. And we’ll run a survey to quantify just a little bit more, some of these things. But yeah, [00:36:00] just, it’s taking what are we learning in our CSAT surveys or in our NPS surveys and is there things sticking out?
Is there more that we need to understand that we don’t understand fully? Yeah. And running the extra research center.
Matt Feeney: Absolutely. Yeah. Identifying pain points and then figuring out how to stop ’em. Yeah. What it all comes down to. Yeah. So the idea behind this was to basically spark some ideas for you, right?
Put some ideas for in your head of how market research can answer some questions that you might be dealing with in your specific business. Just to wrap up, if you’re looking to start utilizing market research more prevalently within your business we would say that the first the first step would really be to just assess what your knowledge gaps are, right?
Figure out what you don’t know that you can actually action against. Define the priorities too. Because as we saw, and we talked about some things that you might perceive as extremely important, might not necessarily be, it might be something that’s nice to add down the road. But in the short term, you want to actually be able to deliver on some ROI related to that research.
And then just at an overall level, I would say start planning research with your end [00:37:00] goal in mind. We always start with our objectives, right? We circle back to it. We make it very common on the research services team to always restate the objectives. At the survey level and throughout each step of the process to make sure that you’re not just going off in the wrong direction inadvertently, you’re always bringing it back to that larger question that you need answered.
So I think that’s it. Thank you all for very much for your time. Thank you everyone.
