Inside the Ivory Tower

23 Apr 2024 | A statement is not fact

In May Contain Lies, I highlight the value of academic research. While it’s far from perfect, it can be more reliable than practitioner studies for a number of reasons:

  1. Its goal is scientific inquiry, rather than advocacy of a pre-existing position or releasing findings to improve a company’s image.
  2. It’s conducted by those with expertise in conducting scientific research.
  3. Papers published in top scientific journals are peer-reviewed, which helpsimprove their accuracy.

However, authors, journalists, and practitioners will sometimes cite research as if it bears the hallmark of academic rigour when it actually does not. This is not their fault; they may not understand the intricacies of academia since academia itself is rather non-transparent. This post aims to demystify two aspects of the ivory tower that are often misunderstood.

1. What Does It Mean For A Paper to be Published?

The word “published” is used to mean two quite different things that are often conflated.

1a. Public Release

One is the public release of a paper. After working on a paper for several months – sometimes, years – the authors will finally decide that it’s good enough for public dissemination. They can release it in several (non-mutually exclusive) ways:

  • On their own website
  • On a centralised website, such as the Social Science Research Network (www.ssrn.com), a hub for papers in the social sciences. Anyone can post a paper on SSRN.
  • Through a working paper series. These are run by research organisations and restricted to their members. Examples in my field are the National Bureau of Economic Research, the Centre for Economic Policy Research, and the European Corporate Governance Institute. Only Fellows or Research Associates of these organisations can contribute to these series. The organisations will put the paper on their website, and may have a mailing list where they notify subscribers of new papers. Central banks and regulators also have their own working paper series.

These papers are known as working papers. Academics would never call them “published”, even though they’re public, because they are merely working drafts that have not been peer reviewed by anyone. They will typically present the paper at conferences and other universities, and send them to other scholars for feedback, before submitting them to an academic journal.

You often hear references to a “paper published on SSRN” or a “paper published by the prestigious National Bureau of Economic Research”. Readers often take this as a sign of credibility, but the paper has not been checked by anyone, and SSRN and NBER are not scientific journals. The papers have been made public, but not published (at least not in a scientific sense).

“A paper published by Harvard University” is particularly misleading. It implies that it has the seal of approval of Harvard University, but Harvard doesn’t publish studies. Anyone with the loosest affiliation to Harvard can post a paper on SSRN or their own website without Harvard’s approval – a tenured professor, a part-time lecturer with no research duties or a master’s student writing a thesis. (This contrasts with organizations which do require institutional sign-off, so it makes sense to refer to a McKinsey study.) Sometimes, companies will say “our paper coauthored with Harvard University”. Often it actually means that they found one person affiliated with Harvard who was willing to do a paid consulting project with the company. Harvard neither conducted nor endorsed the research.

The Financial Times Responsible Business Education Awards, which does an excellent job of showcasing academic research with practical applications, nonetheless lists “Federal Reserve Bank of Chicago”, “Trinity Business School”, and “Energy Institute at Haas” as journals in their table of the winners. But they are not journals; these entries should either be blank or replaced with “Unpublished” as they give the impression that the papers have been peer reviewed.

1b. Journal Publication

The second meaning is the publication in a scientific journal. This involves submitting it to peer review. The journal’s editor sends the paper to leading experts in the topic, who submit their confidential views on its suitability for publication. At the most elite journals, 90-95% are rejected. The remaining 5-10% aren’t accepted immediately either; instead, their status is “revise and resubmit”. The editor and reviewers highlight concerns for the authors to address, and the paper can still be rejected at the next round. It’s not unusual for a manuscript to take five years to be published after its first draft – a hard slog for the authors, but essential to give readers confidence in the results. As discussed in the Introduction of May Contain Lies, a study reached completely the opposite of its original conclusion after it went through peer review and corrected its mistakes.

Peer review is imperfect. Mistakes are made; sometimes flimsy papers get published and good papers get rejected. But it’s better to go with something vetted than something unvetted. Published in a peer-reviewed academic journal is very different from being “published” in the sense of being made public.

In short: check if a paper is published in a scientific journal, rather than simply made public.

2. What is a Professor?

Peer review is valuable, but slow. It takes many years for a paper to go through peer review. A paper could be accurate but not yet published because it’s new. Thus, another thing to look at is the credentials of the authors. Often people highlight that a study was written by a “Professor” as if this is another hallmark of scientific accuracy. But, just like “published”, people use the term very loosely and it means a whole range of things.

2a. Tenured/Tenure-Track Professor

This is a professor whose main job is scientific research (although she will also teach courses). She’s hired on the basis of her scientific research credentials and evaluated principally on the quality and quantity of her scientific research output. There are different ranks:

  • An Assistant Professor is untenured, but on the tenure track. After a certain number of years, they are evaluated for “tenure” based principally on their research output. If they are granted tenure, they have permanent employment.
  • An Associate Professor is the next rank up. At some universities they are tenured; at others they are untenured.
  • A Professor is the highest rank, and always tenured. Sometimes the Professor will have an endowed chair, in honour of a person or company who donated money to fund that position, e.g. the “John B. Neff Professor of Finance” or “Nippon Life Professor of Finance”.

The rank is second-order when it comes to research credentials. Some of the most groundbreaking research is produced by untenured Assistant Professors. The scientific community will refer to all of them as “professors” even though only the third has the rank of Professor. I will refer to all three ranks as “research professors”.

2b. Adjunct/Clinical/Practice Professor

This is a faculty member whose main job is teaching rather than academic research. They may write informative articles on business (e.g. for the Wall Street Journal, Forbes, or Harvard Business Review), but not conduct research to be published in scientific journals. An Adjunct or Clinical Professor will typically have another main job (e.g. Managing Director at an investment bank) and be teaching at the university part time. A Practice Professor will teach at the university full time.

The distinction between (2a) and (2b) is absolutely not to say that one is better than the other. They are simply different. Adjunct/Clinical/Practice professors typically have vastly more real-world experience than research professors. They are also expert at disseminating knowledge (including the translation of academic research by others into simple language) through books, newspaper articles, and blogs; they write excellent opinion pieces or review articles that do not aim to be scientific research. However, when it comes to conducting scientific research, it is research professors who have the expertise.

2c. Visiting Professor

This is a particularly ambiguous title, as it encompasses two types. One is a research professor who is on sabbatical from her home institution and visiting another university to get fresh ideas and forge new research collaborations. A Professor at Harvard could be a Visiting Professor at Yale during her sabbatical. Such a visiting professor should be viewed as a research professor, i.e. category (2a).

The second is typically an adjunct/clinical/practice professor who teaches very few classes; at many institutions, you need to teach a minimum number of hours to have an adjunct/clinical/practice title. Some may not even live in the same country as the university. They are similar to (2b), i.e. may have substantial practitioner experience but not necessarily expertise in academic research. Thus, it’s essential to check if they have a “home” institution, e.g. the Visiting Professor at Yale’s home institution is Harvard.

2d. Non-Professors

There are a worrying number of faculty members who call themselves “professors” when they are not any type of professor. They may have positions such as Lecturer or Visiting Lecturer, which is also a teaching position. Some of them overexaggerate their credentials and a tell-tale sign is including “Professor” in their LinkedIn profile name (most actual professors do not). They also write on LinkedIn that their job title at various universities is Professor when the university websites list it as Lecturer. One of them I first encountered when he asked me a question during the Q&A of a conference and made a point of introducing himself as “Professor John Smith”; most people would say “Hi, my name’s John”. People who fall over themselves to stress how they are professors are often not.

In short: check if a “professor” is a research professor if wishing to assess the reliability of scientific research that s/he has authored.

Inside the Ivory Tower

Inside the Ivory Tower

In May Contain Lies, I highlight the value of academic research. While it's far from perfect, it can be more reliable than practitioner studies for a number of reasons: Its goal is scientific inquiry, rather than advocacy of a pre-existing position or releasing findings to improve a company's image. It's conducted by those with expertise in conducting scientific research. Papers published in top scientific journals are peer-reviewed, which helpsimprove their accuracy. However, authors, journalists, and practitioners will sometimes cite research as if it bears the hallmark ...
Does only 2% of VC funding go to female founders?

Does only 2% of VC funding go to female founders?

A widely quoted statistic is that only 2% of VC funding goes to female founders. For example, this Forbes article highlights that "only 2% of all VC funding goes to women-led startups" and asks "Why is only 2% of VC funding going to female founders"? If true, this statistic is substantial underrepresentation and needs to be urgently addressed. However, it's problematic for several reasons. 1. The Statistic Ignores Diverse Teams The 2% statistic actually refers to companies founded solely by women. It ignores diverse companies founded by both men and women. This is strange, because ...
An unhealthy obsession with organisational health

An unhealthy obsession with organisational health

Two leading asset management firms drew my attention to the McKinsey Organizational Health Index as a potential tool to evaluate a company. A book, "Beyond Performance 2.0: A Proven Approach to Leading Large-Scale Change", written by two McKinsey partners, claimed that companies with high scores on this Index trounced their unhealthy peers along a range of performance measures. For example, their shareholder returns were three times as high. But as I wrote in an earlier post, rather than being more impressed by big numbers, we should be more sceptical. If it were really possible to ...