The Marshmallow Study revisited

23 Sep 2023 | Evidence is not proof

One of the most famous psychology experiments conducted on children is the Marshmallow Study. In a 1972 paper, Walter Mischel and co-authors gave three-to-five-year-olds at the Stanford Bing Nursery School a marshmallow. They could eat it now, but if they waited 15 minutes, they’d get a second marshmallow.

A series of follow-up studies tracked the kids over time. It found that those who were able to wait for the second marshmallow ended up doing better in life — higher SAT scores, higher educational attainment, lower body mass index, less drug use, fewer mental health issues, slower to frustrate, and many other desirable outcomes. That makes sense — if you can delay gratification, you’ll study harder, exercise more, and resist vices. These studies are seen as hard evidence on the value of self-control, and led to marshmallow tests and self-control techniques being integrated into school curriculums. Sesame Street’s notoriously gluttonous Cookie Monster appeared in a video where he has to wait to eat cookies and learns some strategies to help him stay in control.

But data is not evidence, because it may not be conclusive — there may be rival theories. Waiting for the second marshmallow might not reflect self-control but something else:

  • Affluence. Perhaps poorer kids eat whatever food’s in front of them — and that’s rational because they’re not sure when the next meal will come from. There might be food in the cupboard today, but it might be gone tomorrow so waiting is risky.
  • Family background. Kids with a difficult home life have learned not to trust others — if their parents constantly break promises (either due to untrustworthiness or financial necessity), then they can’t be sure that the experimenter will bring the second marshmallow.

It could be affluence or family background that drove the kids’ future success, not self-control. If so, teaching children to delay gratification without changing these other drivers might make little difference. Indeed, it could backfire, if it teaches kids to be naively trusting when they need to grab what they can, or diverts resources away from other initiatives targeted at the real causes, such as free school meals.

In theory, you’d solve the problem by controlling for affluence and family background. But in practice, the Stanford researchers couldn’t. The experiments were on children of Stanford faculty and staff, so there were few differences in wealth and upbringing that could be controlled for.

So a separate set of researchers took a much bigger dataset (900 rather than 90 kids), and crucially one containing a variety of backgrounds: over half had mothers who hadn’t completed college. For these kids, the link between waiting for the second marshmallow and academic outcomes at age 15 was only half as strong as in the original studies — and it fell by two thirds (to a sixth of the original size) after controlling for family background, early cognitive ability and the home environment. Moreover, there was no correlation — even without controls — with behavioural outcomes at age 15.

This is an example of why evidence is not proof: it may not be universal. What you learn from 90 kids of Stanford faculty and staff might not apply to other children. This warns us about taking the findings from one specific study and using them to influence educational policy across the country as a whole.

Was Rachel Reeves British Girls U14 Champion – and does it matter?

Was Rachel Reeves British Girls U14 Champion – and does it matter?

"I am – I was – a geek. I played chess. I was the British girls under-14 champion." So said Rachel Reeves in a Guardian interview. The Independent, Sky News, and Reuters ran similar headlines: They claimed that her chess ability would lead to her becoming a successful chancellor. Reeves herself said "I've learned to think ahead, to plan a strategy." The BBC turned it into a broader political narrative about sexism and private schools: "Growing up, Reeves was driven around the country by her parents to compete and beat privately educated boys, often as the only state ...
Ultra-Processed Evidence

Ultra-Processed Evidence

Chris van Tulleken’s “Ultra-Processed People: Why Do We All Eat Stuff That Isn’t Food … And Why Can’t We Stop?”, on the dangers of Ultra-Processed Food (UPF), was one of the most successful books of 2023. It was a #1 Sunday Times Bestseller and an Economist, Daily Mail, Good Housekeeping, and Amazon Book of the Year. Even now (January 2025) it’s still #9 in the Amazon charts, having spent 88 weeks in the top 20. Due to its popularity, I’d heard about it before reading it, and the punchlines resonated strongly with me. I try to eat home-cooked meals wherever possible, consume very ...
The study that never was

The study that never was

In March 2022, my LinkedIn feed lit up with people sharing this article: It is by Reuters, a highly-respected news source, on a report by Arabesque, an asset manager and data provider I greatly admire and wrote about in Grow the Pie. It was taken at face value and shared uncritically by LinkedIn Top Voices, such as the post below with 415 Likes and 92 comments: One of these comments, by another LinkedIn Top Voice, claimed that "Diverse boards do better period" and that this evidence could be used against the "non converted". The "period" implies that the issue is open-and-shut, ...
No results found.