A long-running tracker has an obvious advantage: history. But what happens when the methodology changes and the numbers move with it Chuck and Maggie use the University of Michigan’s Survey of Consumers as a jumping-off point. After the study transitioned from phone to online data collection, the consumer confidence index showed a notable drop between the two approaches, even though the broader trends looked similar. That raised a difficult question: was consumer sentiment really changing that much, or was part of the movement being introduced by the methodology itself? It's a useful case study for anyone managing a long-running tracker. Methodologies eventually need to keep pace with changes in how research is conducted, but protecting years of historical data makes those decisions complicated. Chuck and Maggie talk through how researchers can approach that tradeoff - from running parallel waves and watching for changes in sample composition to considering whether outside factors, like an election or economic uncertainty, could make the results harder to interpret. What does it take to maintain a tracker as methodologies, consumers, and business needs change?
Key Takeaways
Quotes
Resources
Maggie Bright 0:00
Hey everybody! This is Maggie Bright from Murphy Research, and you're listening to Key Takeaways. In today's podcast, Chuck Murphy and I are going to talk about trackers and the best ways to maintain them over the years. Hope you enjoy it. Hey Chuck, how are you?
Chuck Murphy 0:23
I'm good. How are you?
Maggie Bright 0:24
I am actually really excited too to talk about trackers today. And it's not often that I drive around in my car thinking about trackers, but I was driving home the other day, and I was listening to the radio, and there was a story that came on about how consumer confidence and consumer spending metrics that have typically kind of tracked together in the most recent release of one of the studies are now not tracking so together. And so I thought, hmm, what's going on there? Is this a methodological thing? And I kind of started diving deep into what could be the discrepancies between the two, and it really got me thinking about you know trackers and maintenance and all of those types of things. So I'll give you a little background if you want it, and then we can talk about more practical things. But essentially, there is a consumer confidence is measured by two different studies. One of which is the University of Michigan's Survey of Consumers. Full disclosure to anyone listening: Hi mom, I am a University of Michigan alum. Love the school, and it's going to be hard for me to say bad things about it. So Chuck is going to do all of that. But in 2024, the University of Michigan switched that survey from a phone study. So they like had to fire the last phone house in existence, and switch it to an online survey, and so a lot of the the reading I've done and the you know sort of the information that's come out is that this methodological change is a big reason for the the the two scores sort of diverging. So consumer spend, which is measured by the things we buy, the goods we buy, and consumer confidence, which is a consumer stated measure. So all of that got me thinking about kind of tracker maintenance and those types of things, and I wanted to get kind of your take on all of this. But a pause there for a second because I just word vomited a lot of things about University of Michigan and consumer confidence, but yeah, are you familiar with the study? Do you want me to talk more about it? Kind of, what are your first thoughts?
Chuck Murphy 2:32
Well, you know what's funny about this is like I feel like I feel like this is a study I've heard about for my whole life. It's been since
Maggie Bright 2:40
1940. It's it's old.
Chuck Murphy 2:41
1940,
Maggie Bright 2:42
yeah.
Chuck Murphy 2:44
I've never delved that deeply into it, though. To be honest with you, it's one of those things you hear reported all the time. I see it reported, but it's not something I ever really looked at closely. The methodology of how they did it.
Maggie Bright 2:58
Yeah,
Chuck Murphy 2:58
I'm
Maggie Bright 2:59
no expert on it, but it's it's a phone survey that historically I think was 900 to 1000 people monthly. It's a random dial, and it asks some. I think I actually don't know how long the the full survey is, but it comes out with sort of like these key metrics around you know how confident are you about sort of the state of the economy and about various goods that you might be buying and sort of your long term outlook as well as like the immediate what's happening now?
Chuck Murphy 3:29
It's very heavily weighted study, right? I mean, the Fed looks at it when they're setting rates. Like it's it's a. I actually didn't realize it's been around since 1940. That's amazing. It it it
Maggie Bright 3:38
is, and I think what's so interesting about what's happening right now with it is there are a lot of articles that are like should the Fed be using this? Is this you know is this the right metric and/or should the scores be adjusted because of this methodological change? And so it's brought to you know forward a lot of questions about kind of how do you ensure that your tracker is relevant? How are you maintaining it? And there are other studies out there that measure very similar things that have always been that are newer and therefore have always been online, and they are not seeing sort of these dramatic fluctuations in their numbers the way that University of Michigan study
Chuck Murphy 4:24
is, I think I think it's funny. There's there's a lot of different parts about this topic we could talk about, and I'm going to start with the part that I'm probably least qualified to talk about. But I never gave a lot of thought to how the Fed does what it does until last year, when Trump started arguing with the Fed and saying they were politically motivated, and then it sort of came up in the news a lot last year in a way that kind of made me think more about it, which is it's kind of funny because I think obviously it always comes up. Times of more turmoil, right? Like the Fed came up a ton in in 2007, the eight, but I think at that time, for whatever reason, I don't know, it just was it didn't it didn't capture my attention the way it did last year, where I started thinking about is like how do they do this and like the whole idea of like how do they set dot plots and what do they do, and I actually think it's very interesting that this is how they do it, and there's. I think obviously, for most of this podcast, we'll talk about market research and tracking and methodology and things that are more in my wheelhouse. But one thing that got me thinking about is if you call someone up and you you ask how confident they are, I don't think consumers. I mean, we kind of know this, right? They're not that forward looking with stuff like this, it would be more methodologically sound to just track what they actually spent in the last 30 days. I think, like I don't, I don't know that like asking them for a forward-looking metric is going to change. There's not going to be that much of a delta between what they spent in the last 30 days and what they expect to spend in the next 30 days. I don't think. I mean, maybe I'm wrong on that. Maybe they are really good forecasters, but it would seem a lot easier to just track actual spending, and that opens up other ways to to measure this that that might be more efficient and easier.
Maggie Bright 6:11
They do, I think. And again, I am not an economist, but the I think this consumer confidence number is typically tracks nicely with consumer spend, and that's what's so strange about what's happening now is that they're starting to diverge more. If we look at the University of Michigan numbers, and it's it's sort of like the you know it's interesting. There was an article in the Wall Street Journal just a couple of days ago that was like, "Are we really this gloomy? Like, is consumer sentiment this low? And if so, why is spending still strong? And I think that you're right that you know as the landscape has gotten more complex and more sort of polarized, I do think it's more challenging for consumers to give a good description of like what's going to happen next because there's more uncertainty than there might have been in, I feel like 1940 may not be the best metric, but because a lot going on then too. But you know, like there's there. I do. I agree with you. I just think it's too. It's it is really hard to predict the future, and it may be getting harder for a lot of consumers to do so. But
Chuck Murphy 7:19
yeah, and I wasn't even making a political like the polarization thing. That's actually like a separate topic in my mind. But I'm just more saying just like in terms of we don't do we really need these two studies? Like you, you would just I would think you would just look at spending. Could I mean I don't know, but I know that
Maggie Bright 7:36
that's market research. Like,
Chuck Murphy 7:39
but yeah, I know that the more interesting topic to research is is kind of how you when you when you you know first talked about this the the the thing that's interesting to me and we should we should talk a little bit about the methodology they're using here changes they're trying to make because as market researchers it's very relatable right but it is there is a a lot of to consider when you want to change. Like I've never been involved in the tracker that's been going on since 1940. I'll tell you that we have this same conversation with with clients sometimes. They're like, "Hey, I've had this track. I actually just had a conversation on that this morning. Hey, I've had this tracker. It's been up for a long time. I want to reevaluate how we do it. Yeah, and that's it's tricky. It's it's always very very tricky, right? And there's a there's a lot of resistance to change because of situations like this, right? You know, to I'm gonna even though I'm not an expert in this, I'm just gonna go ahead and say that when they changed this study, they they ran into some problems in terms of trackability.
Maggie Bright 8:43
Yeah, and I think I mean it. In in a very public way, they ran into some problems, right? Because now all of these different, the Chicago Fed, the Wall Street Journal, other you know researchers from Stanford and Yale and all that are calling into question if the change is real, or if the change is driven by the methodological change, and like so, that's very problematic for them because it's like called their this entire study that's been around since 1940 into question, and in addition to that, they're having all these people tell them what they should do to make the survey right now, and I think you know, and to you know, University of Michigan has come out and said, no, it's not the methodology. This is a reflection of, you know, we've put some corrections in there, and this is a reflection of the economy. But it's not lining up to what other studies are saying. And so, yeah, like they ran into a lot of problems, and I think it is fascinating on a lot of levels. One, because we don't normally talk about survey methodology in mainstream media, right? Like when I heard I was listening to NPR or something, I was like, "Wait a minute, no one talks about this. Like they're talking about methodological changes, which is fascinating. And then the other thing is, you know, I think that it calls into question something that has been historically very stable and reliable. Which is you know every time I see polling on news and things like that like I'm always looking in the footnote to see base sizes and things like that and it I mean it's amazing that how where the data could it could be true right in terms of like they collected results and they put out information but it may not be a you know true reflection of what's actually going on because there's something flawed about how the data was collected or reported, and so I found it fascinating to think about. Okay, what what does the methodology do, or what does a change in the methodology, or an inability to keep something up to date, like it could really steer a strategy wrong or a you know a long term decision wrong if you're not paying attention to all these little mechanics that are going into collecting that data over time. So got really wonky there. Sorry. Yeah.
Chuck Murphy 10:46
There's a there is funny. There's there's we're kind of talking about two different things in an overlapping way. Yes. I think to at least in my perception, the two things at hand are okay. We've got this consumer confidence study, and what did they do? And then I think at a more general, broad thing is like how do you handle, you know, changing a tracker or or maintaining a tracker, keeping it current, keeping the methodology current. So maybe as a what I would suggest is the first step is let's just spend like 30 seconds talking about what they were doing, what they changed, and and and put that in context because I actually think for a lot of researchers. This is really interesting to me that this is this you know. So let's you want to just cover the methodology quickly, and then we can here we could go back to some general kind of ideas that come up when you when you think about changing the track. Love
Maggie Bright 11:32
it. See, you got to keep me in line here. I'm all over. So as I mentioned, I think right around 1940 they started this survey. As far as I know, it's always been a phone survey. I don't believe there was anything that happened prior to phone, but essentially they would randomly call people, do the survey online, and then they decided to switch to an online methodology, which the transition occurred in 2024. According to everything I've read, they were sort of studying how to do that transition for many years prior. But in 2024, they were running things in parallel. So they started out where 75% of the data collected was phone and 25 was online. The next month it was a 50/50. The next month it went to 25 phone 75 online and then the final month it was like 100% online. I may have my timing slightly off, but they did a phased sort of phased one in and the other out, so that now it is 100% online. And the one of the other things that changed, I believe, and again that it was approximately 900 to 1000 completes per month when they were doing phone, and it is much larger than that now that they're online. And that was one of the key reasons for going online. In addition to sort of, you know, trying to transition to the what everyone else is doing with surveys, and the what they saw was during that transition time, while the trends between the phone and the online were pretty similar, the actual consumer confidence index, which they I believe they use a couple of different metrics to get to that index, dropped pretty dramatically, and I think it's about a nine percentage point drop month over month, when they between the two methodologies, and there are the two or the sort of three explanations that I saw for the drop. One was people are more negative online than they are with phone, and they're less engaged in the process because they're doing it online and independently. There's this vibes story. So essentially, people are feeling really downbeat about the economy, but it's not affecting their spending yet. And some of that is driven by this K-shaped economy, which I will not get into. And then the third one is that sentiment is being muddled by politics. So essentially, Democrats are feeling less positive than Republicans during the Biden administration. It was flipped, so depending on who the president is, that drives some of your sentiment. So those are sort of the three explanations for why this has happened. But I think there's a larger sort of discourse about kind of like, you know, yes, the the vibes and the political piece may be driving some of this, but let's really talk about sort of this methodological piece and can this still be trusted?
Chuck Murphy 14:29
Yeah, it's funny because there's like seven different topics in here that are all kind of fascinating and and interesting, you know, as as a social scientist and a researcher. But I think to to stick with the methodology Thing, I find this is really interesting to me because one is this is a side note. I doubt they started as RDDU.
Maggie Bright 14:49
You are correct that this started as an in-person methodology. Oh, did you look
Chuck Murphy 14:54
that up? Yeah, that would a door would make a lot more sense because it was
Maggie Bright 14:57
not formalized in night. It was not actually formal. As the sort of the Michigan survey until 1952, but in 1940, and again, this is all my quick Google research, so or someone will correct me at some point. It was actually so one of the original researchers on it was Likert, which tells you how old that seriously. Yeah, It was during World War II. Wait, okay, hang on. Let me go back. So the first surveys-it was like the first surveys of consumer attitudes in finance were commissioned by the Federal Reserve and the U.S. Department of Agriculture, and then they started in World War II asking consumers about their attitudes towards war bonds and inflation, and so it sort of emerged
Chuck Murphy 15:47
around inflation. That would make sense, right? Because especially after what happened in Germany, they would be hyper hyper aware of the yeah. That makes a lot of sense.
Maggie Bright 15:58
Connection was that they concluded that consumer spending depends heavily on both ability and willingness to spend, and so that's the the the sort of high. And it's like how much you spend depends. You know, you can measure it based on your ability and willingness. And they
Chuck Murphy 16:14
had just that out after the mistakes of of 1930 s when they yeah that that all makes sense. So
Maggie Bright 16:21
yeah. So not until 1952 did they begin to formally do it quarterly, and then it went to monthly later.
Chuck Murphy 16:32
Yeah,
Maggie Bright 16:32
turned into this index of consumer sentiment. So
Chuck Murphy 16:37
yeah, it's really interesting because it is, and it is, it's it's funny because they're they obviously are coming at this from an academic perspective and and not only just an academic perspective but but but a very highly visible academic study that's used I mean right there with maybe the census is what as a super super important yeah and and
Maggie Bright 16:56
talk about a nice long trend line so you can absolutely understand why they would be unwilling to interrupt that to move it to to introduce methodological to it.
Chuck Murphy 17:10
Right. No, it it totally makes sense. But so maybe maybe to kind of circle back around and and say what could we learn from this as brands because I do think there's some you know in the world we're in there's a lot of things here that these are always like very tricky issues where you know a lot of times people are either they're kind of they're either super resistant to change like that's been going a long time I don't want to touch it which just seems a little bit like how this was. If they talked about it for seven years, which is kind of amazing, and again, I totally get it because it's been around for a long time. But what happens with that is there's a little bit of a decay, right? There's a decay in terms of are you using the latest methodology? Are you getting the best? You've you've kind of got this deferred maintenance problem where you thought for so long about fixing it that you've really gotten behind the eight ball. Yeah. On the other hand, you don't want. We also see sometimes people that you know. Hey, I want to go with a person I know and like, so I'm just going to switch this big study that's been going forever really quickly to my favorite vendor. You know, you can you see both sides of this where it's it's it's tricky, but maybe we talk a little bit about some of the things to think about in terms of how to manage these trackers, which are obviously, you know, they're usually among the most important studies run.
Maggie Bright 18:29
Absolutely, or at least the most consistent and available studies run. Like they, they're sort of like this this through line that goes through a lot of the other research that you're doing. So you see like something changes in the tracker, and then you have other studies that sort of tell you the why. But the tracker alerted you to the fact that it's happening. But yeah, I mean, I think one of the one of the questions that I hear our clients asking a lot, or I hope that they're asking, is is the current tracker serving the needs of the organization anymore? And I think that's really the jumping-off point for a lot of these things. It's like, yeah, maybe we have this trend line, or we have this thing that we've had forever, and the question is less about like, ooh, are we going to change things, and more about like, is it actually doing what we need it to do anymore? Is it answering the right questions in the way that is actionable for our organization. I think that's usually like a nice starting point where it's like you know, if Michigan had perhaps asked that question sooner, they would have been like, "Hey, everyone else is doing it online, and we're still doing it by phone. Maybe that's a problem that we need to think about or address. So, and I'm guessing they were asking that question all along, but I think there's a real like just keeping something for the sake of keeping it rarely serves an organization. So it's like, what is this? What does this need to do? What is it doing now? And what's the difference between the two, so that we can figure out what to change and how to change
Chuck Murphy 19:53
it? Yeah, and it's tricky because I think this obviously varies a lot by category, right? Like there, there are categories that are evolving. Super fast, but you do some of the questions you just mentioned offhand are so important, right? Like it's funny. We used to. I feel like you know things are more just available now. But we used to. I have to have this list of like questions you should ask every year about your tracker. Are there new competitors in the space? Are there new like benefits or needs or things that are being kind of strategies or tactics that are being evaluated that that might make sense to add or there things that are dated and you want to take out right like but you do have to look at the content of these and then you have to look at the methodology right and the methodology is really tricky to change but you could see what happens if you if you don't make changes and and it's it's very tricky to manage this and figure out the right way to navigate it without every change does bring some unwatered baggage, right? If you can change the results or make things less trackable, but it also there's also baggage that comes with not
Maggie Bright 21:01
yeah
Chuck Murphy 21:01
maintenance and upkeep,
Maggie Bright 21:04
I mean I think one of the things that the Michigan approach did right was running these parallel waves as they were thinking about how to change things, which which makes a lot of sense. And I know that's something we would recommend if our clients were thinking about making dramatic changes. Is sort of like, hey, keep the old one for a while. Let's try let's try the new one and see where where we net out on on the two to figure out you know what's changing, why is it changing, how are things differing between the way you used to do it and the way you're thinking about changing it too. And so I think that like it's I think that's really important as you think about making dramatic changes to a to something like this. Is like, hey, let's run them both at the same time, see what the differences are, because there there are going to be differences, right? Like you could run the exact same questionnaire with two different panel sources and do everything exactly the same, and there will be differences, right? So you know there are going to be differences. It's just are those differences things that make sense that you can explain that are acceptable in the context of the business decisions you're trying to make, so I mean I think that's a really important piece of the puzzle is running things. It's the
Chuck Murphy 22:14
parallel waves, like obviously, you know, with any methodology change, I feel like parallel waves used to be almost automatic when you were changing something like a tracker, and I feel like there's a big resistance to that sort of thing these days. And I'm not sure how to phrase why. I
Maggie Bright 22:34
mean, I can tell you what I think is why it's time and money, right? It takes time and to be honest. By the way,
Chuck Murphy 22:40
one of the things I would say is, 15 years ago, it was more time and more money.
Maggie Bright 22:45
Yeah, agreed, agreed. I think it was just that perhaps the time is the bigger issue there. Like we have to make decisions so much faster, and people aren't patient. I also wonder, and this is purely hypothetical on my not hypothetical, but like I'm just gonna speculate here. The I think that people are more comfortable making decisions with less certainty. I don't know why that is, but I think sometimes that you know it used to be like if I have 80% of the data, that's you know that's good enough. But I I wonder sometimes if if people are just sort of like there's less. I don't want to say less rigor because I don't think that's it. But sort of like they're they're more willing to to make a decision without the full set of data. Sometimes that's a hard to
Chuck Murphy 23:33
be honest with you. This I never really thought about this. I don't know why. As I think about it now, I feel very confident that we do a lot less parallel waves than we would have done 15 years ago. I'm not sure I have a strong hypothesis. I mean, maybe maybe my first hypothesis would be that there's a lot more data sources now. You know, like you, there's a lot more internal data. There's a lot more things trackable, right? You
Speaker 1 24:00
know,
Chuck Murphy 24:01
And so maybe it's something like that where the the the brand equity tracker was kind of the end all be all 15 years ago, and now it's it's it's still on the dashboard, but it's next to other metrics. I don't know.
Maggie Bright 24:16
There's more ways to cross reference your data to see if it's correct versus doing parallel waves,
Chuck Murphy 24:22
yeah,
Maggie Bright 24:23
that could be. It is.
Chuck Murphy 24:25
It's kind of the best in class way to do this, right? And it, yeah. I, I, you said it briefly, so I'm not sure until you got it, but it sounds like this Michigan one may have to maybe. They did a slow transition in methodology, but not necessarily parallel waves. They were
Maggie Bright 24:42
they did a slow like a stepwise transition and they were comparing the data as they went to see that the trends were the same. I think, and again, I would need to spend a lot more time to say this really confidently, but I do think that there were some different. In the sample composition between the phone and the online, so even though sort of the trends were similar when you started to break things down, there were it skewed ironically the online one skewed a little bit older, it skewed a little bit more democratic. So that's one of the other reasons that maybe there were these big shifts in the in the overall sentiment. So, despite doing parallel waves, it looks like they ended up with different sample compositions to some extent between the two modes. Yeah,
Chuck Murphy 25:31
and it sounds like from the story again, I'm not an expert on this, but it sounds like from the story that even when they they kind of, I'm sure they kind of weighted back and and accounted for the sample changes, and there's still some issues that they can't explain. Yep. Which is always the risk in these sort of things, right? It's always like as as someone on our side of the business is always you're always like, oh man, I please don't give me a big spike right before or after a change, right? Like we're just like, oh geez, how am I going to explain this? It's always it's always like the the tricky part of this.
Maggie Bright 25:59
Well, I think too, like 2024 was you know like it's an election year. It was coming out of I mean we were out of COVID by then, but there was a lot going on in 2024 as well. So it's like a sort of a turbulent time to do a method a big methodology change too, just due to sort of external factors. Yeah. So like you know another thing to consider is like what is the environment that you're in which you're doing this, and does it make sense? You know, is it relatively calm or is it really turbulent? And do you want to make that change in that environment is important?
Chuck Murphy 26:33
Yeah, and I mean the tricky thing about some of these changes is that it is there's no way around the fact that it is a change, right? You cannot you cannot get the exact same sample when you change methodologies, and I think that just requires very clear communication, right? Even if you do parallel tests, like you should you should be clear about what changed exactly when, and you know, just kind of let people have all of the details early and quickly. Like, hey, this is this is going to be a big change. You can resell it a little bit. Here's why we're making the change. Here's why we need to do it. But it's and sometimes it goes super smooth and everything works exactly as you expect. And sometimes you get these spikes, and sometimes it's really hard, especially if the spike seems to be legit and caused by something else. But you're not sure if the whole thing is related to the change. That's that's a little bit of what they've got going on here, right? Where they've got like four different things contributing to this change, and it's hard teasing out how much is each one, right? Yeah, because you really know there's these these different things fed into it.
Maggie Bright 27:38
Well, and I think every every sort of academic, you know. Again, I have not read the full body of work on this, but there are a lot of different academic researchers who are saying, "Well, this is why, and that's why. And I mean, so even amongst like a very academic community, they are having a hard time. You know, everyone's come up with with you know measurements around correlations of this, that, and the other, no one no one can agree on exactly what is going on in terms of the data set itself. So even among very researched and trained people, there's no consensus around the the true change. And again, I mean you know in while 2024 may seem like a really long time ago in a tracker that's been running since 1940. Like it may be that it, you know, it kind of balances itself out over time, and that this just looks like a little blip 10 years from now when we look back. So,
Chuck Murphy 28:40
it would be fun to do a study like this and have complete freedom to just redesign
Maggie Bright 28:47
it. Well, you know what's interesting is I think that that has happened, right? There are companies out here that have taken parts of the Michigan study and have done a different, you know, have said, "Okay, we're going to do it all online. Started it more recently, and now those studies are being like, but our data is different than your data. So, like, they took the the questions and the sort of methodology behind the Michigan study, recreated it in an online, more modern environment, and now they're able to say like, your data doesn't match our data, and so it's kind of a weird role reversal where they sort of like, you know, the one that pioneered it is now the one being criticized.
Chuck Murphy 29:25
That's interesting. It is it's very interesting, and it is you know the the part of this that where everything has become politicized is very difficult to tease out, and and It's it's funny because it's we see it in other areas too, right? Not just spending. We we see this come up. There there's a lot of topics now that quickly become politicized, and people just feel like there's a right answer and a wrong answer. They they're not actually telling you necessarily what they think, but more what they think they're supposed to think. It's it's kind of you know it's it's really it's it's becoming very tricky to do stuff like this. But the thing about spending is that just seems like such a great behavioral metric that you could just kind of lock in. Um, I don't know. It's it seems like there's I don't know any of the new studies that I
Maggie Bright 30:18
totally agree that there's a that the behavioral metric is locked in, and it should drive a lot of it. But there's also this idea of like explaining some of it, right? And I think that's what the the nice thing about pairing the behavioral with the attitudinal is that they were able to say like, oh, you know, consumer confidence is starting to wane, therefore spending should start to go down at some point. So it became a little bit of a predictor as well as sort of a reflection on okay, how are people feeling? And the study actually goes into it looks at sort of like durable goods versus you know non durable goods and things like that, and tries to measure if confidence is more you know stronger in certain categories versus others, and then it obviously looks at things about cohorts and household income and things like that. So there's a lot of like texture to it that I think is really interesting, and it and it made a lot of sense to pair the two up and look at them. But if they and I will say, even the ones that don't have methodological changes are they're seeing less of a correlation between the two between the stated data and the behavioral data now. It's just a you know there's even less so with the Michigan data. So even the other sets that are using a more sort of modern methodology are struggling with this issue as well. And so then the question becomes like do you decouple these things, which is sort of where you're headed? Like, do you stop using consumer sentiment as a, as a, you know, sort of like a barometer for future spend?
Chuck Murphy 31:50
I think right now, again, this is not my area of expertise, but I, but I think right now, like, but the reason a lot of ink is being spilled is that sentiment is far lower, far more negative than actual numbers would indicate. And there's I've read so many articles or heard so many people talk about this in different ways, but people aren't sure. Is this politics? Is this like AI fear? Is this you know you know? And it's funny because you you see you see so many people just write about this from their perspective, right? You know, the the younger people will write about how like housing is impossibly unaffordable, and you know, it's there is it's interesting. There's, I think, a lot of this comes down to social media and this this bad news spreads fast. You know, if it bleeds, it leads, or whatever. Back in the old local news days, that people are kind of bombarded with these negative takes on things. Whereas, if you're objective and look at the data, like you're doing better than a new generation ever before you, right? Like this is time. This is really good times. You're rich. You're really
Speaker 2 33:00
rich.
Chuck Murphy 33:00
And but it's just really hard for people to to see that or feel it because something about the way they consume information now.
Maggie Bright 33:08
Yeah, I'm I'm sure that's a a part of it, right? Like it is the the modern news cycle is just more negative or volatile or fast, or prolific. I mean, it's all of those things, and so it's kind of like it creates this this anxiety around. And there's
Chuck Murphy 33:28
so much there's so much anxiety being spun up about so many things, and it makes it makes it like on one hand, it's real, right? So it's like, oh yeah, consumer seven is really low, but it's not necessarily trackable to that same low sentiment reading 30 years ago, right? Like
Maggie Bright 33:48
which
Chuck Murphy 33:49
may have been caused by something much more, uh, whatever real was the wrong word, right? Much more, much more severe, right? Yeah, this was this was a war breaking out, or this was just
Maggie Bright 34:02
volatility in all of it, right?
Chuck Murphy 34:05
Viral negativity.
Maggie Bright 34:07
No, I think that's I think that's fair. Yeah, I mean, it's it's. I think the way that we interpret attitudinal data has gotten more complicated because there's just more. I don't know if there's like more attitudes. That's probably not the right way to think about it. But you know, there's just a lot more going on, and so people are having a harder time processing that information into a nice little. This is how I feel about the future, or you know, this is why I feel that way because there's so much noise going on and how we make decisions and driving that anxiety. So it would, it in some ways makes a lot of sense that brand consumer sentiment and spend are starting to decouple a little bit because we're you know like because we're facing all of these different things that we're seeing in the media. So I mean I don't think it's like that crazy. I think it's just interesting to think about the mechanics behind the two and why they're. Separating is it really just you know because of society or there other structural design things that are driving them apart? So it's kind of an interesting academic exercise.
Chuck Murphy 35:10
Speaking of academic exercises, I'm going to use that word as jumping off point. I'm going to circle right back around to trackers.
Maggie Bright 35:16
I love it.
Chuck Murphy 35:17
Nobody in this world wants to talk about trackers more than me. One one thing that I think about sometimes, just as like a mental exercise that is fun in this scenario, is if you were to start this over today, what what what would be the first thing you would? I mean, I wouldn't say question you'd ask, but the first like slide you would want in your presentation. Like, what is the most important one? And you kind of rebuild it from there, and how would you do it differently? I think is a very interesting exercise to go through every couple years with these things because they do they do take on a life of their own, a legacy, so to speak. That could sometimes be, you know, you could end up with with half of half of them, half of the content being unnecessary at this point, because somebody doesn't know who owns it or why it's why it's fire. So I think that in addition to like this broad-based methodology, just just like, hey, if you were to if you were to start over this, where would you start? What's the most important thing? And you know where that need this is is funny. You know, in terms of our world, our little market research world, the area where that most needs to happen in my mind is friggin' NPS. Like there, there are. I can't believe just as as I mean, I I love all these brands that do the stuff. I I love you. I want you as clients. Yes, but what I can't believe as a consumer how ridiculously far we've taken these dumb NPS things, and and that I shouldn't I shouldn't say it like that, right? Like maybe Nicole make me sound more positive, but it's AI me into saying that in a more professional, positive way. But I mean, it's just it's gotten it's so dumb, and it's just like this box that they feel they need to check because someone told
Speaker 1 37:00
them,
Chuck Murphy 37:01
you know, 1520 years ago that that was what they needed to focus on, and the focus on it is-I actually don't disagree with that idea. Like, if I'm willing to tell you how much I love a brand, that that that speaks volumes, but it's gotten so sloppy, and and it just it you know just it just feels like those those dumb tags on the bottom of every couch cushion that you're not allowed to rip off because of some class action lawsuit just right it's just like this doesn't serve any real purpose anymore no one's reading this thing like this in a in a real way
Maggie Bright 37:40
I mean I think that's a fascinating point because it goes back to what we were saying. Like, what of at the very beginning is kind of like who's using this and what are they doing with it, right? And I, I think sometimes NPS gets stuck in there because no one said like we don't even use this anymore. You know, I, I, I don't. It's, it's funny. I feel like not often does someone asks us to build, like we want to, you know, I haven't heard NPS in a while, right? So I feel like it's becoming a dated, you know, concept that that even you know researchers today aren't thinking about. But it's probably stuck in a lot of people's like that metric is probably still developed on a lot of dashboards that no one's looking at at all anymore, and it's just because it's like a residual that's stuck in there.
Chuck Murphy 38:25
That's exactly why I think it needs to be looked at again. Because it's funny. Here is the time. I feel like this is this. I lived this whole transition, right? Because I remember when this was new, and I remember when every every week some different company was like, "I gotta have NBS, and it started as like lots of custom solutions from people like us, and the the original. It's funny that I do think there's been papers written about how the original findings were flawed in a couple different important ways. But the idea does make sense to me. Like there's as a consumer, there's nothing I could do that is more of an endorsement or brand than tell my friends and family to use it. Yeah, that is. I totally agree with the idea, but I think what happened is people got sold on these these cool solutions that were customized to their business or whatever, and then over the ensuing years, they basically swapped them out for cheaper alternatives, right? Like, oh, there's this company that will you know take my you know I don't want to name names, but take this cost you know in cut 30% off, and then three years later, there's another version that cuts 30% off. That they've got these cheap cookie cutter solutions, but they've gotten they've gotten so bad. They're like one of those old neglected brands that everybody ate at lunch, and nobody that worked there was paying attention. Yeah, like you're so because I I just I notice it all the time as a consumer. Actually, it's funny. I just noticed this yesterday, and I'm definitely not going to name the name because I love them again, and they're a you know good client. But I they sent this thing to me as a consumer, and I was just like, "How did someone okay this? Like it just just kills me because it's like so it it goes against the exact standards, you know when M. This was first. I don't know why I'm going off on this soapbox on NPS.
Maggie Bright 40:03
We went off on NPS last last podcast too. So I'm
Chuck Murphy 40:07
getting seen on. I know.
Maggie Bright 40:09
I think you feel very strongly about the. I do feel the NPS.
Chuck Murphy 40:13
It violates a certain. If you go back to the original NPS sales pitch, the the strongest business case story was was Southwest right Southwest basically threw out the playbook and started over in a customer centric way and they they actually did a great job of that by the way I'm actually fascinated by whether they're going to be able to if that culture will survive these current transitions they're going through but but that's a separate topic but but they sort of threw out the playbook, and then they they kind of reimagined things in a customer centric way, and and that led that was you know delighted consumers, and they were and and now you have everyone doing the exact same NPS playbook, and it's so it's not even formatted correctly. On even I have an iPhone, that's not like I'm an unusual consumer. Like I'm like, come on, it's just gotten so lazy. It's become this check-the-box thing. That's the exact antithesis of why it was created. Yeah,
Maggie Bright 41:06
yeah. No, I I think there's probably a lot of old legacy stuff out there that's like that too. And and you're right. If you just ask simply ask the question, like, if I were going to rebuild this, what would what would it look like? And then I mean, I think a lot of times you can't start over, right? Like, um, but at least you can figure out what you need and what you don't need, and make a more efficient version of your existing. And it is,
Speaker 1 41:29
it is,
Chuck Murphy 41:30
you know, it does, it does relate to the topic here, right? Like that whole idea of like every year, whenever I should start this over, what would I do, and how would I do it? There is something to be said for that. For even traditional brand, I mean, for all of these different versions of trackers we have running, is like, is there a better, more efficient way to do this that that gets people involved and being honest with you, which is the most important hurdle to clear, right? Like, you have to have the right people answering the right questions.
Maggie Bright 41:59
Yeah, I mean, I think one of the things that we typically build into our tracker approach is at a very at a minimum sort of an annual you know come together to talk about here the I mean the other thing I think is interesting depending on sort of the cadence of the the reporting and and discussions like a lot of times you sort of lose the macro trends because you're looking at all these little micro trends you know, looking at all these month-to-month data points, and you forget to look at the whole year. So one of the things that we always recommend is like, hey, let's have this annual get together where we talk about what are the big macro trends. But a big piece of that is like, hey, let's go through and see what are you actually using, who is using what, what kind of questions are you getting back about it, are you getting any questions back about this, and that helps you sort of fine tune what the next year should look like, or what trends you should be watching. You know, like a lot of times there'll be, you know, we've inherited trackers where there's these brands that are still in the in the brand list that haven't been around for five years, and you're like, why is that brand still in here? Like, not on shelf; it doesn't exist, you know. So even like those little tiny things that are really important, and I mean, I think a tracker is a it's a logistical giant if you're running it correctly, right? I think about some of the ones we have running and the amount of time that our staff spends on just like you know the upkeep part of it, as you mentioned, sort of this idea of like our brand list accurate? Are we defining the categories correctly? Are the right people taking it? That type of thing. That's a lot of work, but like it's really important work. And if you're not kind of reviewing all that work on an annual basis, you're probably missing some opportunities to be more efficient and accurate.
Chuck Murphy 43:37
And you didn't even talk about all the sample challenges. These I did.
Maggie Bright 43:40
Yeah, I figured that was an entirely different podcast, right?
Chuck Murphy 43:43
I used to have a colleague that used to say these are these are living living objects. These trackers, right? They always have to they have to be maintained and pruned. And there's there's a lot of truth to that. That it's like it's it's you've got to constantly breathe. You
Maggie Bright 43:57
know, if you'll remember this, but many many many years ago, when I worked at a company that no longer exists, so I think I can tell this story. We had a mail tracker, mail m a i l not m a l e. That was it was very large. It was, I mean, I think we sent 10,000 out a month or something like that. And I remember when we first met, I was on the client side, you were supplier side, and I showed you the paper version of it. And there was a mistake in the scale. Do you remember this?
Chuck Murphy 44:30
I don't remember this. It was a. It was. It
Maggie Bright 44:33
was negative on both ends. It was. Oh, I do
Chuck Murphy 44:36
remember that. I do remember that.
Maggie Bright 44:38
And I, I have no idea how long it was going out like that, but it was like fascinating. You know, I mean, like even even you know, and imagine how much that flood that data set for the amount of time that it was going out. So even you know, even even big big dumb mistakes need to be need to be caught on a given by
Chuck Murphy 44:59
these. To be, these need to be scrubbed every year.
Maggie Bright 45:02
Yeah,
Chuck Murphy 45:04
that's probably a good place for us to wrap up, huh?
Maggie Bright 45:07
Yeah, talking about my past failures.
Chuck Murphy 45:10
No, no, you know, it's funny. It does that just seems like so long ago.
Maggie Bright 45:17
It was a lifetime ago, but yes, greatly. We had to
Chuck Murphy 45:20
order. We had to order, however many 1000s of yeah. No, that that that was Xerox machine for
Maggie Bright 45:28
days. I'm just kidding.
Chuck Murphy 45:30
It's so it's so crazy.
Maggie Bright 45:32
Ah, well, thank you for indulging me on you know sort of this wonky academic trip to get to tracker maintenance.
Chuck Murphy 45:42
I know, I know. This was this was fun-a deep dive in our typical wandering fashion with a lot of people. Not
Maggie Bright 45:49
all those who wander are lost, Chuck.
Chuck Murphy 45:52
Oh wow, Confucius wrapping up this podcast with wisdom. Yes, knowledge.
Maggie Bright 45:58
Yes, I try. I try. All right. Well, until next time, then. Thank you. All right.