Experimenting on WEIRD people

27 Oct 2023 | Evidence is not proof

One of my research fields in behavioural economics, which suggests that people don’t always act in perfectly rational ways. A famous experiment in this field is known as the Ultimatum Game. There are two players, a ‘proposer’ whom we’ll call Amelia, and a ‘responder’, Bilal. Amelia is given £10 and is then asked to suggest how to share it with Bilal. If Bilal accepts the split, both players receive Amelia’s proposal; if he rejects it, both get nothing. It’s rational for Bilal to accept any offer of at least £0.01, because anything is better than zero. Knowing this, Amelia should only suggest £0.01, keeping £9.99 to herself.

But experiments show that Bilal typically rejects any split under £3. He views such a division as unfair and rejects it to punish Amelia, even though it means he also gets zero. Fairness seems to matter even more than putting food on the table.

The first findings were from an experiment on University of Cologne economics students in 1978 (that was published in 1982). One concern with experiments is that they might be unique to a specific setting — students, rather than working professions, and Cologne instead of Colorado. Indeed, the ‘replication crisis’ in social science has arisen because some landmark findings have been overturned in other settings. But the Ultimatum Game has been widely replicated. A study co-authored by Nobel Laureate Alvin Roth repeated it on cities as far afield as Jerusalem, Ljubljana, Pittsburgh, and Tokyo. It’s also been documented in five-year-old kids, and even chimpanzees.

Yet as I dug deeper into this topic for a behavioural economics paper I was writing, I discovered the results weren’t actually so universal. Psychologist Joseph Henrich tested the Ultimatum Game on the Machiguenga people of the Peruvian Amazon. Proposers made miserly offers, and responders gratefully accepted pitiful amounts without viewing them as unfair. As Henrich explained, “Rather than viewing themselves as being ‘screwed’ by the proposer, they seemed to feel it was just bad luck that they were responders, and not proposers.”

Quite why Machiguengans play the Ultimatum Game differently from others is unclear. It’s tempting to dismiss them as being selfish for making low offers, but that would be imposing our own notions of fairness. Machiguengans might equally call it selfish to think you deserve a share of someone else’s good fortune, and to be so jealous of it that you’ll punish them by rejecting a split. Regardless of what causes these differences, they exist. Henrich later wrote  a book pointing out how studies of human behaviour focus on WEIRD subjects — those who are Western, Educated, Industrialized, Rich and Democratic — and their results may not extend to the rest of the world.

In the Ultimatum Game, Amelia gets the £10 as a pure windfall — it drops from the sky like manna from heaven. But what if she had to work for it? Other researchers ran an experiment where two kids fished for magnetic cubes out of different containers. The game was rigged so that some cubes were less magnetic and fell off the rods more easily, but the children didn’t know this. As a result, several pairs ended up with uneven spoils. The catch was combined, and the kids were asked to suggest how to split them. German children thought it’s fair for the one who caught the most to get the lion’s share. But to kids from the #Akhoe Hai//om community in Namibia, fairness meant an equal split — regardless of how much each child contributed.

This is an example of why evidence is not proof. As I wrote in Grow the Pie, “A proof is universal. When Archimedes showed that the area of a circle is pi times the square of its radius, he proved this not just for circles in Ancient Greece in the third century BC, but for circles in Modern Greece today and for circles throughout the world. But evidence may only apply to the country or industry in which it was gathered.”

Even if you have the world’s most rigorous evidence, the results might not extend to other settings.

If at first you don’t succeed, try try again

If at first you don’t succeed, try try again

One of the papers that I cited most prominently in Grow the Pie is "Corporate Sustainability: First Evidence on Materiality". It shows that ESG doesn't always pays off: firms with high ESG scores don't beat the market; only those that focus their ESG efforts on issues material to their industry. For example, climate change is a serious global threat, but it isn’t the most important concern for a tech company that conducts its business in the cloud rather than along the coastline. Thus, a tech company that’s best-in-class in its carbon footprint doesn’t beat the market; instead, it ...
Want a more innovative conclusion? Innovate the conclusion

Want a more innovative conclusion? Innovate the conclusion

'Want a more innovative company? Hire more women'. The title hooked me immediately. I’m an avid follower of the @TEDTalks Twitter page, but I don’t have time to watch every talk. But when I saw one with the title ‘Want a more innovative company? Hire more women’, I wanted to hit play instantly.
You couldn’t even make it up

You couldn’t even make it up

Confirmation bias leads us to make up excuses to dismiss facts we don’t like. If our favourite politician gets elected and the economy tanks, we’d argue it would have done worse had she not been in charge. Or we’d protest that we need to wait another year before we can truly evaluate her performance.