The Hawthorne effect refers to a phenomenon where people change their behaviour if they know they're being watched.
Credit: Hoxton/Paul Bradbury
Careless citations don't just spread scientific myths – they can make them stronger
How misconceptions persist and proliferate within the scientific literature.
21 October 2019
Hoxton/Paul Bradbury
The Hawthorne effect refers to a phenomenon where people change their behaviour if they know they're being watched.
Science, in theory, is self-correcting. But, as a new study demonstrates, some scientific ideas appear immune to criticism. Striking them down only seems to make them more powerful.
The study, published in PLoS ONE by Kåre Letrud and Sigbjørn Hernes of the University of Applied Sciences in Lillehammer, Norway, looks at citations and mis-citations of three articles critiquing the so-called Hawthorne effect. It’s the first in a series of planned investigations of what Letrud refers to as “tenacious scientific myths”.
Similar to the placebo effect, the Hawthorne effect describes a phenomenon where people’s behaviour changes when they are aware that they are being observed or are part of a study.
Researchers investigating the effectiveness of an intervention are often concerned that the Hawthorne effect, rather than the intervention itself, may explain any changes they are seeing.
The Hawthorne effect owes its name to a series of investigations conducted in the 1920s and 30s at the Western Electric Company’s Hawthorne Plant in Illinois. The researchers observed that productivity increased among the workers who were selected to take part in the programme.
However, as Scandinavian health researchers Gustav Wickström and Tom Bendix argued in a 2000 paper, there are numerous alternative explanations for those original findings.
The Hawthorne effect, they concluded, is “ambiguous and disputable” and it is “questionable whether the term has a function any longer in the evaluation of results from intervention research”.
Letrud and Hernes retrieved the full text of 196 papers that cited this critique. The overwhelming majority of these – 155 in all – mis-cited Wickström and Bendix as having affirmed rather than questioned the validity of the Hawthorne effect.
Bendix, now a Professor Emeritus at the Center for Rheumatology and Spine Diseases at Rigshospitalet in Glostrup, Denmark, says he is “embarrassed” for the many authors who have misconstrued his critique.
“l am amazed that our paper has been cited so often,” he says , “but even more amazed that the majority of authors evidently has read it very superficially – if they have read it at all.”
Letrud and Hernes observed similar patterns for two earlier, highly cited critiques of the Hawthorne effect. A 1992 paper by Stephen Jones concluded that “there is slender or no evidence of a Hawthorne effect” in the original studies. Yet of 140 retrievable citations, 60 cited it the paper as affirming the effect.
Likewise, a 1978 critique by Richard Franke and James Kaul with 277 retrievable citations was mis-cited 189 times as affirming the Hawthorne effect.
Notably, these two earlier critiques were initially cited correctly, but over time were increasingly mis-cited.
Letrud and Hernes also suggest that, in many cases, mis-citations occur because the authors citing the papers have not actually read them or have read but misunderstood them.
They note that readers who accept the criticisms of the Hawthorne effect may choose not to mention the effect or the criticisms at all in their own research papers. That would bias citation patterns for critiques or negative evidence towards mis-citations.
But is the Hawthorne effect really a myth? Much of the criticism has focused on the original studies, which were not designed to test the effect. Other researchers have since conducted more carefully controlled studies to test its validity.
A 2014 systematic review found evidence of “small, statistically significant effects” in randomized control trials. The authors concluded, however, that there is “no single Hawthorne effect”.
Letrud acknowledges that the term ‘scientific myth’ is open to interpretation, but a wider concept that includes unwarranted or vague half-truths and misconceptions may be more useful than a hard distinction between truth and falsehood. “In the context of research, the persistent and indiscriminate distribution of falsehoods and half-truths are problematic in very similar ways,” he says.
Read next:
How researchers can improve the quality of systematic reviews