Did the New York Times make Wordle harder?

28 Oct 2023 | A statement is not fact

Many forms of misinformation have a clear source — authors or journalists misquoting a study, or a famous person giving an extreme quote. That allows us to check the underlying source, as in this example.

But some misinformation is in the form of an urban legend, which spreads through the grapevine with no clear source. After The New York Times bought Wordle, many people claimed — quite vehemently, given the addictiveness of Wordle — that the NYT had unfairly made it harder. This may have been driven by confirmation bias, since many players disliked the NYT’s political stance. They made increasingly strong claims based only on their own (or others’) subjective assessments.

Without a clear source for the difficulty of Wordle, you can’t check a person or a study. But, if you’re willing to get your hands dirty, you can investigate the underlying claim itself. Carnegie Mellon computer science professor David Andersen dug into the underlying source file behind Wordle to download the actual word list. In what’s below, it’s the ‘main.4d41d2be.js’ file.

He then ran a Python script comparing the old and new lists, and found this:

What does this mean in plain English? That the NYT removed exactly six words (fibre, lunch, agora, pupal, slave, and wench) and added no new ones, so Wordle became slightly easier, not harder.

Now the person on the street can’t be expected to go through the steps of a Carnegie Mellon computer science professor and conduct the checks ourselves. But what we can do is to see if there’s any evidence behind a claim before believing it, and certainly before sharing it — the third and final tip in my TED talk was to ‘pause before sharing’. Yet if we’re biased against the New York Times, we’re willing to believe claims that it made Wordle harder based on no more evidence that a few people saying ‘I think it’s become trickier’. As this Twitter thread explains, this was a great example of how confirmation bias causes conspiracy theories to spread.

Inside the Ivory Tower

Inside the Ivory Tower

In May Contain Lies, I highlight the value of academic research. While it's far from perfect, it can be more reliable than practitioner studies for a number of reasons: Its goal is scientific inquiry, rather than advocacy of a pre-existing position or releasing findings to improve a company's image. It's conducted by those with expertise in conducting scientific research. Papers published in top scientific journals are peer-reviewed, which helpsimprove their accuracy. However, authors, journalists, and practitioners will sometimes cite research as if it bears the hallmark ...
Does only 2% of VC funding go to female founders?

Does only 2% of VC funding go to female founders?

A widely quoted statistic is that only 2% of VC funding goes to female founders. For example, this Forbes article highlights that "only 2% of all VC funding goes to women-led startups" and asks "Why is only 2% of VC funding going to female founders"? If true, this statistic is substantial underrepresentation and needs to be urgently addressed. However, it's problematic for several reasons. 1. The Statistic Ignores Diverse Teams The 2% statistic actually refers to companies founded solely by women. It ignores diverse companies founded by both men and women. This is strange, because ...
An unhealthy obsession with organisational health

An unhealthy obsession with organisational health

Two leading asset management firms drew my attention to the McKinsey Organizational Health Index as a potential tool to evaluate a company. A book, "Beyond Performance 2.0: A Proven Approach to Leading Large-Scale Change", written by two McKinsey partners, claimed that companies with high scores on this Index trounced their unhealthy peers along a range of performance measures. For example, their shareholder returns were three times as high. But as I wrote in an earlier post, rather than being more impressed by big numbers, we should be more sceptical. If it were really possible to ...