Want a more innovative conclusion? Innovate the conclusion

2 Jan 2024 | A statement is not fact

The title hooked me immediately. I’m an avid follower of the @TEDTalks Twitter page, but I don’t have time to watch every talk. But when I saw one with the title ‘Want a more innovative company? Hire more women’, I wanted to hit play instantly. My most-cited paper is on corporate culture, and just one month ago I’d appointed the first female editors to the board of the Review of Finance, a leading academic journal — the first appointments I’d made after taking over as Managing Editor earlier that year.

The speaker described how her research team investigated the link between diversity and innovation. Like all good TED talks, she takes you on a journey, claiming they started out with skepticism on whether they’d find anything at all:

To put it mildly, we were not optimistic. The most skeptical person on the team thought, or saw a real possibility, that we would find nothing at all. Most of the team was rather on the cautious side, so we landed all together at ‘only if’, meaning that we might find some kind of link between innovation and diversity, but not across the board — rather only if certain criteria are met, for example leadership style, very open leadership style that allowed people to speak up freely and safely and contribute.

Their boldest hope was to find that diversity works in specific situations. But as the speaker continues her journey, there’s a twist which makes her findings even more striking:

A couple of months later, the data came in, and the results convinced the most skeptical amongst us. The answer was a clear yes, no ifs, no buts. The data in our sample showed that more diverse companies are simply more innovative, period.

This was music to my ears. But I wanted to make sure that I’d understood the study correctly, and so I listened to the talk again — this time trying to step outside the story and focus on the evidence. Now, my ears pricked up at something else.

To measure diversity, we looked at six different factors: country of origin, age and gender, amongst others.

In sharp contrast to the title, which focused exclusively on gender diversity, the researchers actually measured six different dimensions of diversity. Even if there was a cast-iron link to innovation, you had no way of knowing whether it was gender diversity, or any of the other dimensions, that was behind it. The title was no more valid than ‘Want a more innovative company? Hire older people’ or ‘Want a more innovative company? Hire younger people’ — but such titles would have likely led to far fewer views.

And the problems didn’t just stop at their measure of diversity, but also of innovation. The most widely accepted yardstick is the number of patents a company produces, as a patent is only granted if something is truly innovative. If you are about quality, not just quantity, you could measure the number of citations to a patent, the market value of a patent, or how distinct it is from a company’s existing patents.

But the team did something quite different. They measured the share of a company’s revenues that came from new products developed over the past three years. That fraction could capture many things other than innovation. It could be high because you’re not innovative – the products you launched four years ago weren’t that ground-breaking, so they fell away fast and are no longer generating sales. Or maybe you did a bad job of marketing them, or a new CEO came in and decided to change direction. Alternatively, you could be generating sales from new products even if they’re not that innovative — they could be a small variation on an existing theme. Even if the 15th edition of a textbook is little different from the 14th, most people will get the latest one.

Perhaps due to viewer complaints, TED have since changed the title of the talk to ‘How diversity makes teams more innovative’, which addresses the first problem. But it doesn’t solve the second, which is part of the same concern — when a study parades a relationship between two things, check and see what it actually measured. If it’s something different from what the authors claim, then a statement is not fact. (And the new title doesn’t address a separate problem, that there’s correlation but not causation, as covered in the section on why data is not evidence.)

An unhealthy obsession with organisational health

An unhealthy obsession with organisational health

Two leading asset management firms drew my attention to the McKinsey Organizational Health Index as a potential tool to evaluate a company. A book, "Beyond Performance 2.0: A Proven Approach to Leading Large-Scale Change", written by two McKinsey partners, claimed that companies with high scores on this Index trounced their unhealthy peers along a range of performance measures. For example, their shareholder returns were three times as high. But as I wrote in an earlier post, rather than being more impressed by big numbers, we should be more sceptical. If it were really possible to ...
Successful companies are successful

Successful companies are successful

I was struck by the Harvard Business Review headline "Companies that Practice 'Conscious Capitalism' Perform 10x Better". On the one hand, it appealed to my confirmation bias. Conscious capitalism involves serving stakeholders, not just shareholders, in line with my book Grow the Pie and work on purposeful busienss more generally. On the other hand, the effects seemed too big to be true. If conscious capitalists could really perform ten times better than the competition, they'd quickly drive their rivals out of business and we'd only be left with conscious capitalists - just like if a ...
Discernment matters even more

Discernment matters even more

In 2015, 2018, and 2020, McKinsey released a trio of papers claiming that diversity has a positive causal impact on firm performance, titled "Diversity Matters", "Delivering Through Diversity", and "Diversity Wins". These studies make basic errors, as highlighted by Green and Hand (2021) and others, yet were widely cited - perhaps due to confirmation bias. McKinsey have now doubled down in their latest (November 2023) report, entitled "Diversity Matters Even More". They claim that "the business case is the strongest it has been since we've been tracking", that their new report ...