SYLVIE DOUGLIS, BYLINE: NPR.
(SOUNDBITE OF DROP ELECTRIC SONG, “WAKING UP TO THE FIRE”)
GREG ROSALSKY, HOST:
In 2010, the economist John List joined with a couple colleagues to do something that economists don’t normally do.
JOHN LIST: We built a pre-K program in Chicago Heights from scratch.
DARIAN WOODS, HOST:
John built a name for himself devising these super interesting experiments in the real world, like creating this preschool to see what works in education.
ROSALSKY: And he and his team found really promising results from this school. The kids’ test scores shot up. They showed character improvements like greater determination and grit. John and his team were excited about all this.
LIST: So I tried to scale it. I went and knocked on the doors of various policy makers, and each time they would tell me, John, that’s great, but we don’t think your results or your program will scale.
(SOUNDBITE OF MUSIC)
ROSALSKY: This is often an issue – something works in a small experiment, but then when, say, the government implements it on a large scale, it doesn’t work out so well.
WOODS: So John got obsessed with the process of scaling. But when he looked at what had been written about it, what evidence there was already out there, he was pretty disappointed. It was more this kind of art than a science. So he decided to change that with his new book.
LIST: The book is titled “The Voltage Effect: How To Make Good Ideas Great And Great Ideas Scale.”
ROSALSKY: This is THE INDICATOR FROM PLANET MONEY. I’m Greg Rosalsky.
WOODS: And I’m Darian Woods. Today on the show – why do some ideas scale and others don’t?
(SOUNDBITE OF MUSIC)
ROSALSKY: Today, we’re talking with the University of Chicago economist John List about his new book, “The Voltage Effect,” which tries to unlock the secrets of scaling.
WOODS: John List has an impressive resume. He has been the head of the University of Chicago’s economics department. He’s been on the White House Council of Economic Advisers. He’s been the chief economist of Uber. He is currently the chief economist of Lyft. I mean, all of this is not bad for somebody who grew up in a small town in Wisconsin.
ROSALSKY: So I got to say, I learned a lot in this book not just about, you know, economics and scaling, I learned a lot about you, Professor List. I learned that you come from a family of truck drivers. I learned you once worked in a cheese factory, which is also the most Midwestern thing I think I’ve ever heard. Actually, what kind of cheese was it?
LIST: There’s all kinds of cheese. It was called the Wisconsin Cheese Men (ph), and what we did is we packed cheese into gift baskets. And then those gift baskets, every winter, would go out to – all over the world. Look, you can’t beat Wisconsin cheese, Greg.
ROSALSKY: So we’re not here to talk about Wisconsin cheese, as great as it may be.
WOODS: Oh, yeah.
ROSALSKY: We’re here to talk about scaling. But actually, one of my favorite examples in the book does have something to do somewhat with cheese. In the mid-1990s, McDonald’s tried to launch a new cheeseburger called the Arch Deluxe.
WOODS: I’d never heard of it until you told me about this.
ROSALSKY: Yeah, well, it’s been called one of the biggest product flops ever. It’s literally in the Museum of Failure, which is an actual museum.
WOODS: So how do we avoid getting into the Museum of Failure, Greg?
ROSALSKY: So John List identifies five vital signs or five factors that people should consider when thinking through whether an idea, policy or product can actually scale. If McDonald’s paid attention to one of these vital signs, maybe it could have avoided the Museum of Failure.
Back in 1996 though, McDonald’s executives believed that the Arch Deluxe cheeseburger was the solution to their problems. The company’s growth had flatlined, competitors were eating into their sales, and McDonald’s developed the Arch Deluxe to try and reverse this decline.
WOODS: And this burger stood apart from the other McDonald’s burgers. It had this stone-ground Dijon mustard, peppery spice, smoky bacon and a split-top potato bun. It was a little bit more sophisticated than the typical cheeseburger. McDonald’s began testing this burger in focus groups, and its executives loved what they found.
LIST: Now, what they found in those focus groups is that it was a wild success. Those focus groups loved the Arch Deluxe.
WOODS: McDonald’s found that adults in these focus groups especially loved the Arch Deluxe, and that was important to McDonald’s because they struggled to get more adults into their doors. So the company wanted to scale it up.
(SOUNDBITE OF MCDONALD’S COMMERCIAL)
UNIDENTIFIED PERSON: It starts with a full quarter pound of beef…
WOODS: And they spent hundreds of millions of dollars blanketing the airwaves with ads.
(SOUNDBITE OF MCDONALD’S COMMERCIAL)
UNIDENTIFIED PERSON: If it were any more grown-up, we’d need to check your ID. It’s McDonald’s with a grown-up taste.
WOODS: This was the most a fast-food company had ever spent on a product launch.
ROSALSKY: But despite the focus groups that showed that adults loved the burger and despite the fortune the company spent on marketing the Arch Deluxe to adults, sales of the burger proved to be abysmal. The burger did not do well at scale.
LIST: They scaled it, and it turned out to be a deluxe failure.
WOODS: (Laughter) How could this be a deluxe failure? Well, John List says that those focus groups that inspired McDonald’s to scale up the burger – those focus groups were flawed.
LIST: They put out an ad – come and be a focus group for McDonald’s. Try our great hamburgers, and you’re going to try some new menu items. Just step back and say that focus group is probably someone who is either crazy about McDonald’s or loves all kinds of burgers or loves to try new things.
ROSALSKY: In other words, the people who participated in the focus groups were not representative of customers at large. Economists like John call this selection bias. The people who volunteered or self-selected to take part in the focus groups were not typical consumers.
LIST: The average person, as it turns out, would rather just have a Big Mac or a Filet-O-Fish and french fries. They don’t want the fancy stuff.
WOODS: The Arch Deluxe is an example of ignoring one of John’s five vital signs. Now, this is a bit of a mouthful – misjudging the representativeness of an initial population or situation.
ROSALSKY: Another vital sign that John identifies in the book is looking out for false positives. That’s when an idea an organization wants to scale may look like it holds promise, but if they actually rigorously test the idea, they will see that the idea is bogus.
WOODS: And this is common in social science. One study shows something works, but then when someone else tries to do a similar study, the results fail to replicate. John, who’s now the chief economist at Lyft, says that companies should fight this by trying to replicate their results multiple times.
LIST: So at Lyft, we might find a great result in Seattle. That doesn’t mean we should roll that out ubiquitously. We should replicate it in Seattle to make sure it’s a true result in Seattle and then try out a few other markets. The idea is you should constantly be poking and prodding at your idea to make sure it’s a true result.
ROSALSKY: A third vital sign is watching out for unintended consequences. John gives the example of Uber trying to raise wages for its drivers. It did this by raising their base fares. A recent study found that it worked at first, but after six weeks, Uber drivers’ hourly earnings fell back to what they were before the fare increase. Why? Basically, because the opportunity of making more money on the app enticed more people to drive for Uber. That made the market more competitive and resulted in each individual driver getting fewer trips overall.
LIST: And in the end, they’re working more, but they’re driving around with an empty car more often, and they undo all of the good stuff of the rate increase.
WOODS: A fourth vital sign is thinking through the business side and all the costs and benefits of scaling. For example, you need to ask whether your product will have economies of scale. So will the more you produce make the next item you produce cheaper and cheaper?
ROSALSKY: Finally, John says it’s important to evaluate whether what makes your idea work on a small scale can be replicated when it’s deployed on a larger scale. He gives the example of trying to scale his experimental preschool. He says it would have been a mistake to only hire the best and brightest teachers in the experiment and then expect their great results to be replicated when it was rolled out nationwide.
WOODS: Yeah. I mean, it’s a sad-but-true fact that public schools have limited budgets, and their ability to hire tens of thousands of teachers of the same caliber – that’s probably pretty small. And without those high-caliber teachers, policymakers need to consider whether the program’s promising results in a small trial can still scale.
LIST: So the idea with this vital sign is understand when you scale something, what are the constraints that you’re going to have at scale?
ROSALSKY: There’s, of course, much more in the book. Maybe reading it could help you avoid the Museum of Failure?
WOODS: I do not want to end up there, Greg.
ROSALSKY: (Laughter) You know what would be cool? If the Museum of Failure had a cafeteria, and they served the Arch Deluxe there. You know what? Museum of Failure, that’s a free idea for you guys.
WOODS: This episode was produced by Nicky Ouellet and engineered by Isaac Rodrigues. Corey Bridges fact-checked the show. Viet Le is our senior producer, and Kate Concannon edits the show. THE INDICATOR is a production of NPR.
(SOUNDBITE OF MUSIC)
