An unhealthy obsession with organisational health

7 Apr 2024 | A statement is not fact, Data is not evidence

Two leading asset management firms drew my attention to the McKinsey Organizational Health Index as a potential tool to evaluate a company. A book, “Beyond Performance 2.0: A Proven Approach to Leading Large-Scale Change”, written by two McKinsey partners, claimed that companies with high scores on this Index trounced their unhealthy peers along a range of performance measures. For example, their shareholder returns were three times as high.

But as I wrote in an earlier post, rather than being more impressed by big numbers, we should be more sceptical. If it were really possible to triple your returns by adopting some simple practices (or hiring McKinsey to implement them), the unbelieving heathen would be driven out of business. Indeed, one of the asset managers remarked that the supposed improvements in performance were “bold claims indeed!” and asked me if they were too good to be true.

Unlike academic social science research which is freely available for public scrutiny (even if the published version is behind a paywall, a pre-print version is available for free at www.ssrn.com), here you need to buy the authors’ book to verify the claims. Many readers won’t bother, and take the claims at face value because they want them to be true. But some of my own work is on corporate culture, which seems related to organizational health, so I was intrigued and bought a copy. Unfortunately, the promises don’t stack up.

1. Organizational Health is Nothing New

The authors claim to have pioneered the idea of organizational health and shown that it improves performance. But what is organizational health? Here’s the authors’ words:

Health is how effectively an organization works together in pursuit of a common goal. It is evaluated in levels of accountability, motivation, innovation, coordination, external orientation, and so on. A more memorable way to think about health-related actions is that they are those that improve how an organization internally aligns itself, executes with excellence, and renews itself to sustainably achieve performance aspirations in its ever-changing external environment.”

Sounds like plain old good management – working together well, being attuned to market conditions (external orientation), and excellence in execution. It seems to be old wine in new wineskins; perhaps McKinsey was trying to dress up traditional management practices as a new phenomenon, “health”, to get drum up more business. But, labels aside, perhaps the index is still a valid way of measuring company performance, so the only problem is calling it “health”, rather than anything fundamental with the methodology?

Unfortunately not. The big selling point of the book, and the Index, is that the authors claim to find someting separate from performance. Chapter 1 is entitled “Performance and Health”, and contrasts health with performance, arguing that latter is “what an enterprise does to deliver improved results for its stakeholders in financial and operational terms”. But working together well, external orientation, and excellence in execution will likely help deliver improved performance, so the big distinction between performance and health seems a false dichotomy.

Indeed, in Chapter 2 of the book (which describes the methodology), they write “But we suspect there may be a few statisticians, academics, or skeptics out there who’d like to know more about our survey methords or analyses that underpin our definition of orgaizational health. If you are one of thse, please continue reading. If you aren’t, we strongly suggest you skip to the next chapter, unless you happen to be reading in bed and need help falling asleep”, hoping the reader will gloss over the next section. And the next section is over nine pages admitting that all nine components of the Organization Health Index have been extensively studied by prior research, which has found correlations between each component and firm performance.

2. The Measures of Health are Measures of Performance

Let’s again set novelty aside. If you’re a company trying to improve performance, it doesn’t matter if you’re using old methods as long as they work – a baseball pitcher will use tried-and-tested techniques rather than inventing his own style. Perhaps the value of the OHI is to bring together all the nine determinants of workplace effectiveness under a common umbrella? That provides a handy checklist for executives running their companies.

But the nine dimensions are all so closely linked to performance that any relationship is circular. For example, they include:

  • Direction: A clear sense of where the organization is heading and how it will get there that is meaningful to all employees
  • Leadership: The extent to which leaders inspire actions by others
  • Motivation: The presence of enthusiasm that drives employees to put in extaordinary effort to deliver results

Moroever, they’re captured by surveying employees, and thus prone to halo effects. If the company is already performing well, employees are likely to give high scores in the survey. They’ll say that a successful company has a clear direction and strong leadership; it’s much more motivating to work for a successful company than an unsuccessful one. Simply put, one of the best ways to improve your Index score is to be successful in the first place, and so the Index offers little guidance to managers.

3. There Are Almost No Results

For a book that makes such bold claims, there’s a stunning lack of of detail in the results. For example, the book’s blurb says that “the evidence shows that leaders can more than double their odds of successs – from thirty percent to almost eighty”. I wasn’t able to find this result anywhere in the book. And even if it was there, it’s not clear why leaders should care. Much of the book’s discussion of performance contrasts above-average with below-average profitability. True leaders wouldn’t define success as being simply above average. Under the authors’ methodology, increasing performance from 4.9 to 5.1 out of 10 would count as success, but raising it from 5.1 to 10 would not.

The results boil down to three graphs (there are no tables anywhere in the book), with very little detail on the methodology used to generate them:

  • One studies the link between performance and health across 16 refineries at a single oil company: it considers a solitary company and is an anecdote, not evidence. (There is no statistical significance either as the authors measure R-squared, rather than p-values).
  • Another correlates health interventions with five performance measures: profit per banker at a business bank, profit per banker at a retail bank, increase in tonnage at a coal mine, sales to labor ratio at a retailer, and customer churn reduction at a call centre. These are very partial measures of performance. Wells Fargo increased profit per banker by selling products that customers didn’t need; an increase in tonnage can be achieved by hiring more employees; sales to labor ratio says nothing about the profitability of sales; and customers can be retained by offering them costly sweeteners.
  • The third compares total shareholder return and return on invested capital between the bottom and top quartiles on the Index, without a single control. It could be that small companies are more healthy (it’s easier to set direction and leadership in a small firm), and small companies often outperform large ones. Or, soaring industries are likely to have higher index scores (as there’s a clearer direction than an industry that needs to reinvent itself), and are more likely to enjoy strong performance.

4. The Causal Claims Are Very Strong

Unfortunately, sloppy studies from consultancies (and McKinsey in particular) are nothing new. But even some McKinsey studies admit that “correlation is not causation”, although they typically bury it at the back. The claims in this book are even stronger than the typical consultancy study. They mock those who question the value of organizational health as having “their heads well and truly in the sand as the facts to the contrary are incontrovertible”. They claim not just correlation but also causation, arguing that “We’ve gone even further than regressions to test causality, as well … we felt the case for causation was well and truly closed”. The subtitle of the book refers to a “Proven Approach”, claiming that they’ve got to the top of the Ladder of Misinference when they haven’t even made the first rung.

The authors go overboard in trying to bamboozle the reader with tons of statistics and jargon, to brow-beat her into submission and into admitting that they’ve prove their point. They claim their statistical wizardry is so strong that the humble reader might be disoriented by them; in addition to the earlier quote, they exclaim “Phew! Don’t say we didn’t warn you that this section is only for the statisticians, academics, or hardcore skeptics”. They have various “factor analyses” trying to validate their survey methodology. But, as is often the case in such studies, common sense trumps statistical pyrotechnics. It doesn’t matter what their “Cronbach’s alpha” or “Standardized Root Mean Square Residual” are if the survey is plagued by the halo effect.

 

Inside the Ivory Tower

Inside the Ivory Tower

In May Contain Lies, I highlight the value of academic research. While it's far from perfect, it can be more reliable than practitioner studies for a number of reasons: Its goal is scientific inquiry, rather than advocacy of a pre-existing position or releasing findings to improve a company's image. It's conducted by those with expertise in conducting scientific research. Papers published in top scientific journals are peer-reviewed, which helpsimprove their accuracy. However, authors, journalists, and practitioners will sometimes cite research as if it bears the hallmark ...
Does only 2% of VC funding go to female founders?

Does only 2% of VC funding go to female founders?

A widely quoted statistic is that only 2% of VC funding goes to female founders. For example, this Forbes article highlights that "only 2% of all VC funding goes to women-led startups" and asks "Why is only 2% of VC funding going to female founders"? If true, this statistic is substantial underrepresentation and needs to be urgently addressed. However, it's problematic for several reasons. 1. The Statistic Ignores Diverse Teams The 2% statistic actually refers to companies founded solely by women. It ignores diverse companies founded by both men and women. This is strange, because ...
An unhealthy obsession with organisational health

An unhealthy obsession with organisational health

Two leading asset management firms drew my attention to the McKinsey Organizational Health Index as a potential tool to evaluate a company. A book, "Beyond Performance 2.0: A Proven Approach to Leading Large-Scale Change", written by two McKinsey partners, claimed that companies with high scores on this Index trounced their unhealthy peers along a range of performance measures. For example, their shareholder returns were three times as high. But as I wrote in an earlier post, rather than being more impressed by big numbers, we should be more sceptical. If it were really possible to ...