Academic urban legends
Abstract
Many of the messages presented in respectable scientific publications are, in fact, based on various forms of rumors. Some of these rumors appear so frequently, and in such complex, colorful, and entertaining ways that we can think of them as academic urban legends. The explanation for this phenomenon is usually that authors have lazily, sloppily, or fraudulently employed sources, and peer reviewers and editors have not discovered these weaknesses in the manuscripts during evaluation. To illustrate this phenomenon, I draw upon a remarkable case in which a decimal point error appears to have misled millions into believing that spinach is a good nutritional source of iron. Through this example, I demonstrate how an academic urban legend can be conceived and born, and can continue to grow and reproduce within academia and beyond.
Bauerlein et al. (2010) claim that we are currently experiencing an ‘avalanche of low-quality research’, and academia has become an environment where ‘[a]spiring researchers are turned into publish-or-perish entrepreneurs, often becoming more or less cynical about the higher ideals of the pursuit of knowledge’. Whether the current state of affairs is better or worse than before, it seems reasonable to assume that corner-cutting is an unfortunate side effect of publication pressure and competition for academic positions and scarce resources, especially in milieus where counting publications is more important than reading and evaluating them. In this article, I explore a particular set of corner-cutting techniques that reveal much about strategies of reading, writing, and citation, as well as the development of academic urban legends.
The digital revolution within academia
Twenty-five years ago, it could take weeks to obtain a specific source document needed in order to verify or explore a reference that for some reason had caught one’s attention. As a consequence of the digital revolution, it is possible today to obtain a wide spectrum of sources within minutes or seconds. Formidable databases, advanced scanners, optical character recognition (OCR) technology, and new features of reference management software such as Endnote have made it possible, with some experience, to read an academic text together with the sources it refers to.
For those of us who are old enough to know what a card catalog is, it is outright fascinating to sit in front of a computer with two monitors, reading an academic text on one of them and having the sources it refers to (or perhaps should have referred to) on the other. Having immediate access to most, if not all, of the sources behind an academic text opens up a number of exciting opportunities, but also exposes some unpleasant surprises. In the past few years, it has become dramatically easier to identify cases of plagiarism and scientific misconduct, and also to discover other types of academic shortcuts, and to see how shockingly frequently they are employed.
In this article, I will limit my focus to one specific type of situation that seems to cause problems for a large number of researchers and students and that provides a breeding ground for a wide variety of academic shortcuts. The situation of interest is one that all writing academics will encounter numerous times during their career: when we read a text and find a statement or specific point that we would like to use ourselves, and we discover that it is already accompanied by a reference.
Spinach as a good source of iron
I will illustrate this situation with an example I encountered not long ago, in which a scientific article I was reading presented me with some new and outright fascinating knowledge. The following quote, including the reference, is taken from an article published by K. Sune Larsson in the Journal of Internal Medicine:
The myth from the 1930s that spinach is a rich source of iron was due to misleading information in the original publication: a malpositioned decimal point gave a 10-fold overestimate of iron content [Hamblin, 1981]. (Larsson, 1995: 448–449)1
The quote caught my attention for two reasons. First, it falsified an idea that I had carried with me since I was a child, that spinach is an excellent source of iron. The most striking thing, however, was that a single decimal point, misplaced 80 years ago, had affected not just myself and my now deceased parents, but also a large number of others in what we place on our table.
After reading Larsson’s article, I took a poll of colleagues at my institute, asking them why they think spinach is healthy. The conclusion was quite clear. The belief that spinach is a good source of iron, although falsified 30 years ago by Hamblin in a British Medical Journal article, is still widespread among my colleagues, all of whom have, at minimum, a master’s degree in health sciences. In fact, the history of spinach consumption in the Western world indicates that we are dealing with a decimal point error of enormous consequences. For generations, parents have been wasting their time and energy, nagging their more or less anemic children to eat a vegetable that the young typically abhor, ruining family social events in the process.
Truth be told, there is iron in spinach, but not significantly more than in other green vegetables, and few people can consume spinach in large quantities. A larger problem with the idea of spinach as a good source of iron, however, is that it also contains substances that strongly inhibit the intestinal absorption of iron (see e.g. Garrison, 2009: 400). Simply put, spinach should not at all be the first food choice of those suffering from iron deficiency.
Larsson’s article made me aware of the remarkable fact that a large number of people in the Western world have been misled for a staggeringly long time. Since so many people still believe that spinach is a good source of iron, I have good reason to convey this newfound knowledge to others. The story of this decimal point error is, in addition, a brilliant illustration of how a small stroke may fell a great oak, and a reminder of the importance of accuracy and quality control in the production and distribution of scientific knowledge.
How, then, should I properly pass on the important messages I learned from a single sentence in Larsson’s article? The following seems like a fairly appropriate paraphrasing of the original text:
The idea that spinach is a good source of iron is a myth that was born in the 1930s, due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value.
How should I refer to my source? If I want to include this sentence in an academic publication, what should I place after my sentence? There are several options in this particular case, and I will present the most common among them and discuss what consequences the various alternatives may have.
The first alternative is to leave the sentence as it is, without any reference at all. This is something I can do if I am distributing common knowledge, which obviously is not the case here. Should I choose to omit a reference, I could, in the worst case, be accused of plagiarism, and the most naive among my readers would perhaps think that I was the one who discovered the decimal point error with the dramatic consequences. A more likely outcome would be that my readers would become puzzled, or perhaps irritated, by the fact that I did not provide any form of documentation for how such a remarkable thing could occur. In principle, it should be impossible or very difficult to get undocumented statements of this kind published in scientific publications, but as we will see toward the end of this article, it happens from time to time.
The simple truth is too simple
In academia, the following is fortunately a far more common way of passing on such a message:
The idea that spinach is a good source of iron is a myth that was born in the 1930s, due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value (Larsson, 1995: 448–449).
Here, I simply and honestly refer directly to the source where I found the information, and I am even courteous enough to provide exact page numbers for readers who would like to verify it, or who may be interested in exploring whether there is more to learn from Larsson. The problem in this case is that I omit a piece of information: the fact that Larsson’s statement is based on an entirely different source, namely Hamblin (1981). In other words, I am referring to an article that I very well know is a secondary source, and thus hide from my readers the fact that Larsson actually just passed on information published by Hamblin 14 years earlier. A good reason for avoiding the use of secondary sources in academia is that messages that pass through several links have the unfortunate tendency to become modified or altered along the way, as in the whisper game. My readers will in this case think that Larsson is the primary source, and my statement will therefore look more solid and trustworthy than it actually is.
Providing this type of reference has other negative consequences. This time it is not me, but Larsson who gets undeserved credit for the discovery of the decimal point error. Another consequence is that readers who try to verify my statement will get an unpleasant surprise when they look up the source (Larsson, 1995: 448–449) I have provided. They will then discover that they have become, quite unwillingly, participants in a kind of treasure hunt. Having reached the first post at page 449 in Larsson’s article, their only options would be either to give up the quest for verification, or to proceed to the next destination, Hamblin’s (1981) article in British Medical Journal.
In a case like this, when I am aware of the fact that my citation has weaknesses, it can be tempting to try to make the statement more convincing by adding more references I might have easily available. In our digital age, it is not difficult to find other sources that contain the story about the decimal point error and its dramatic consequences. If the supply is as rich as in this case, it is a good idea to select alternatives published fairly recently in respectable journals, such as the article by Frangoulis et al. (2010: 43), or maybe the book by Carroll and Vreeman (2009: 114). If I want, I can add numerous sources like this, getting a long and impressive list of references, full of prestigious journal names and publishers. This would, of course, be an academic confidence trick, but it would not be exposed until readers took the time to look up the sources I listed. Only then will they be able to see that they are all secondary sources, and that they all refer back to the same single sentence in Hamblin’s (1981) article.
Honesty is not always the best policy
A third and even more honest alternative would be to refer to my source in this way:
The idea that spinach is a good source of iron is a myth that was born in the 1930s, due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value (Hamblin, 1981, cited in Larsson, 1995: 448–449).2
This is a perfectly legitimate way of referring to sources in cases where it is difficult or impossible to obtain a primary source. The 1981 volume of British Medical Journal is, however, easily available for anyone with Internet access. Should this type of reference be used in this particular case, it could reflect a case of academic laziness, but coupled with utmost honesty. Another and perhaps more likely explanation is that we are dealing with an academic who has not understood the importance of the principle of striving to use primary sources in order to minimize the whisper game effect.3
This type of citation does not necessarily have to be explained by laziness or lack of knowledge, but rather by an almost touching degree of confidence and trust. In this case, I put my trust in Larsson that he has read and interpreted Hamblin correctly and that he has good enough reasons for putting his trust in Hamblin. Whatever explanation, I deserve credit for having made it perfectly clear that I have not consulted the primary source, and that my statement is the last and therefore the least trustworthy link in a chain of sources.
The risk of buying a pig in a poke
A fourth alternative, which unfortunately is far more common than we should wish, is to solve the problem the following way, without consulting Hamblin (1981):
The idea that spinach is a good source of iron is a myth that was born in the 1930s, due to a misplaced decimal point, causing the concentration to appear ten times higher than its real value (Hamblin, 1981).
In this case, I am referring directly to a source that I have not consulted myself, and in doing so I am committing an academic lie. The same degree of trust as in the previous alternative is present, but the difference is that the stakes are now much higher. What I hope to achieve with this type of reference is that nobody will discover my laziness. I simply pretend that I have taken the effort to consult Hamblin (1981), without having done so. In short, I have plagiarized the Hamblin reference from Larsson.
An attractive aspect of this academic shortcut is that it is usually impossible to discover and to prove the sin committed. Academics such as Larsson presumably check their sources thoroughly, and double-check that their own text corresponds with the sources it refers to. If Larsson has understood Hamblin correctly, and Hamblin is worthy of his trust, then there would be no negative consequences from this highly dubious type of reference, neither for my readers, nor for the truth and reliability of what I am writing.
Referring to sources that one has not consulted can be, however, a risky business. Academics, as other human beings, do from time to time misinterpret or make errors that are not discovered by peer reviewers or editors, even in respectable journals such as the Journal of Internal Medicine. When several authors independently of each other manage to misrepresent a single source in exactly the same erroneous way, the explanation is either a statistically unlikely coincidence, or a case where authors have plagiarized references. Systematically patterned distributions of errors and misinterpretations are in fact common enough to make it possible to study the prevalence of citation plagiarism and the unfortunate consequences of the practice. Such studies are indeed sad readings for those who are concerned about safeguarding academic principles of honesty and quality control (see, for example, Harzing, 2002; Morrisey, 2004: 152–154; Wetterer, 2006; Wright and Armstrong, 2008).
The narrow path has its temptations
The four alternatives treated so far all represent various types of academic shortcuts, and they all share attributes with various ways in which rumors are spread. The common denominator of all of them is that I do not consult the assumed primary source in this case. I simply end up, in a more or less honest way, passing on what I have read that Larsson has read in Hamblin (1981).
The final alternative is to follow the short and narrow path back to Hamblin’s article to see what he wrote on the issue. It is, of course, wise to check the accuracy of what we base ourselves upon when we write and publish, and there is also the possibility that we might learn something even more valuable about the issue.
In this particular case, there is in fact a lot more to learn from what we have so far assumed is the primary source, not just about the decimal point error, but also about academic shortcuts, and, not the least, about the conception, birth, and growth of academic urban legends. As we shall see, it turns out that Larsson has in fact made several errors when reproducing Hamblin’s message, and on top of it all, Hamblin is not at all to be trusted in this particular case.
We find the following in Hamblin (1981):
The discovery that spinach was as valuable a source of iron as red meat was made in the 1890s … German chemists reinvestigating the iron content of spinach had shown in the 1930s that the original workers had put the decimal point in the wrong place and made a tenfold overestimate of its value. … For a source of iron Popeye would have been better off chewing the cans. (p. 1671)
The myth about the iron content of spinach was not born in the 1930s, as told by Larsson. The decimal point error was made 40 years earlier, but was disclosed in the 1930s, and Hamblin makes it perfectly clear that he himself was not the one who made the discovery. The third, and greatest surprise is not immediately clear from the quote above: Hamblin does not provide a reference to support his claim that a decimal point error actually was made; nor does he give any names, dates, or other information that could help us verify how the error was made and by whom, or who should be credited for its discovery and correction.
I have now taken the effort to consult the assumed primary source, only to discover that the case was considerably more complex than I had anticipated. In such a case, it may be tempting to suppress what I just discovered, that the fascinating decimal point error is thus far an undocumented piece of information, what we usually call a rumor. I could correct my first statement, for example, in this way:
The myth that spinach is a good source of iron has its origin in a decimal point error in the 1890s. German scientists discovered the error about 40 years later (Hamblin, 1981: 1671).
Here, I have done my academic duty by consulting Hamblin directly, and I have corrected my statement in a manner that corresponds much better with what Hamblin actually wrote. If I publish this text, I am still guilty of spreading a rumor, and also of disguising its rather obscure origin through a reference to a prestigious scientific journal. I very well know that the decimal point error story suffers from a lack of documentation, but I cannot resist the temptation to use it, perhaps because it fits so nicely into the argument of an article I am writing.
In fact, despite the vagueness and complete lack of documentation that characterize Hamblin’s account of the decimal point error, the story has been picked up by numerous authors who have redistributed it through journal articles and books, turning it into a full-blown and still blooming academic urban legend.
Realizing that a remarkable account like this lacks documentation, the safest and wisest thing would be to forget the whole issue. A tempting compromise could be to use terms and formulations that communicate reservations or doubt, as David A. Kronick did when he wrote the following, accompanied by a reference to Hamblin (1981):
Typographical errors, for example, do occur. It has been suggested that [emphasis added] spinach got its reputation as a dietary supplement because of a misplaced decimal point in which the iron value was given as ten times higher than it was. (Kronick, 1985: 75)
Kronick deserves praise for including his reservation: ‘It has been suggested that …’, but such phrases tend to change or disappear when knowledge passes from one text to another. Hypotheses, assumptions, and suggestions can in such a way be transformed into knowledge and scientific facts. We shall soon see a spectacular example of how this can happen, and what unfortunate consequences can follow.
An ironic point concerning the two articles we have quoted (Hamblin, 1981; Larsson, 1995) is that they are both written by academics placing themselves at the frontline of the fight against bad science and academic carelessness. In his article titled ‘The dissemination of false data through inadequate citation’, Larsson appears almost furious with the ways in which some scientists uncritically redistribute research findings and conclusions of dubious character. Larsson obviously has an important message, but he manages to do exactly what he is criticizing others for in quite a spectacular way. Larsson’s (1995) next sentence adds even more momentum to the irony:
The myth from the 1930s that spinach is a rich source of iron was due to misleading information in the original publication: a malpositioned decimal point gave a 10-fold overestimate of iron content [Hamblin, 1981]. Once a paper with misleading information has been published, it is almost impossible to stop citation. (pp. 448–449, emphasis added)
How true. There are no indications that the decimal point error ever was committed, and the whole story could have ended there, as a small slip of tongue by Hamblin in the 1981 Christmas issue of British Medical Journal. When the slip has turned into a widely distributed academic urban legend, it is thanks to the effort, or rather lack of effort, of a very large number of uncritical academics, among them Larsson himself.
The conception and birth of an academic urban legend
Hamblin’s article, with the short but telling title ‘Fake!’, is not just being used and cited by scientists concerned with academic quality and honesty. His story about the decimal point error extends far beyond academia, to newspapers, magazines, TV-programs, and web pages, including some that specialize in debunking myths and urban legends.
Very few have questioned Hamblin’s claim that a decimal point error is the reason why many of us still erroneously believe that spinach is a good source of iron, and many surely continue to pester their children as a consequence of this belief. There is, however, one exception.
Mike Sutton is a British criminologist who did not give up after having read Hamblin’s article and, like me, struggled his way through the rather confusing list of references without finding a single trace of any decimal point error. Sutton (2010a: 7) did what we all should do more often in such cases. He contacted Hamblin directly and asked him from where he had learned of the decimal point error. Hamblin replied that he could not remember, but that he was sure he had not made it up.
Sutton (2010b) argues convincingly that there were entirely other reasons, such as contamination during the analysis or the confusion between fresh and dried spinach, that caused the exaggerated figures in the 19th century. He also criticizes Hamblin for perpetuating another related misconception: that Popeye was created in order to promote spinach for its iron content. According to Sutton (2010a: 13–14), Elzie Crisler Segar had an entirely different nutrient, vitamin A, in mind when he invented Popeye and contributed to a massive increase in spinach consumption in the United States during the 1930s.
Shortly after Sutton published his article in 2010, a reader by the name Bonnie Taylor-Blake made Sutton aware that a certain Arnold E. Bender could be the source that Hamblin was unable to recall. The criminologist immediately brought the investigation a step further (Sutton, 2010b). In Bender’s inauguration lecture at Queen Elizabeth College, University of London in 1972, he made the following statement about the iron content of spinach: ‘the fame of spinach may well have grown from a misplaced decimal point’ (Bender, 1972: 11).
Bender repeated exactly the same sentence in a book titled The Facts of Food that was published 3 years later (Bender, 1975a: 15) and in a journal article (Bender, 1975b: 142) that was printed the same year. Two years later, he wrote a short comment letter (Bender, 1977) in The Spectator, where the wording is slightly more assertive: ‘The fame of spinach appears to have been based on a misplaced decimal point’. In 1982, the year after Hamblin, based on something he had heard or read somewhere, stated that a decimal point error was actually made, the following phrase appeared in a textbook for medical students (Bender and Bender, 1982: 55): ‘the belief can be traced back to a mistake in the transcription of analytical results in 1870, when a decimal point was misplaced’.
Bender’s suggestion about a possible decimal point error has now turned into an assertion linked to a specific year, two decades earlier than Hamblin’s purported date. But again, the statement still lacks a reference or any other type of documentation. It is therefore difficult to know if Bender’s (1982) increase in certainty could be a result of his own more thorough investigations on the issue or whether he had been influenced by Hamblin’s (1981) article. A third possibility is that Bender’s gradual change of wording, consistently without any precise documentation, could be a consequence of a common human weakness which allegedly has a particularly high prevalence among anglers telling stories about their past catches.
Bender and Hamblin have not just provided us with examples of how even top-level academics can sometimes be careless in their attitude to references and documentation. These publications between 1972 and 1982 have also given us an excellent opportunity to study the birth of an academic urban legend and to explore the micro dynamics behind the often dramatic outcomes of the whisper game. The decisive moment was most likely when Bender’s ‘appears to have been’ became replaced by Hamblin’s assertive statement. In other words, Hamblin had certainly heard about the decimal point error from someone, but accidentally turned Bender’s suggestion into a piece of fact, which then was blessed with the stamp of reliability associated with British Medical Journal.4
What could be more appropriate in this article about academic carelessness than to present a positive example, highly worthy of imitation? Terence Hamblin, the man who made the undocumented and most likely non-existent decimal point error known to the world in 1981, submitted the following comment to Mike Sutton’s web page almost 30 years later:
Thanks for pointing out my mistake of 29 years ago. I never could remember where I had first seen the decimal point story – I thought it was in Reader’s Digest. I am very pleased to see that you have uncovered the whole story and very willing to admit that I was wrong. Incidentally my name is Terence not Terrance. (Sutton, 2010b, ‘comment to the article’)
Completion of the circle
It turns out that Reader’s Digest has in fact published the story of the decimal point error. The following quote is from a book titled Facts and Fallacies: Stories of the Strange and Unusual, published in 1988, 7 years after Hamblin made his unfortunate statement in British Medical Journal:
How Popeye got it wrongPopeye the Sailor has done more for spinach than any salesman could ever have dreamed possible. When Popeye made his first appearance in the 1930’s, spinach consumption in the United States rose by 33 percent. Why? It was spinach that gave Popeye his mighty strength and bulging muscles, because it was so full of iron. But the belief that spinach promotes strength was based on a very simple mathematical error. Nutrition researchers in the 1890’s put a decimal point in the wrong place, thus giving spinach 10 times more iron than it actually contains. (Reader’s Digest Association, 1988: 264)
Popeye was not wrong at all, but Hamblin was, and with him a formidable flock of academics and others – among them Reader’s Digest Association – who uncritically followed in his footsteps. The decimal point error that Bender hinted about, and which Hamblin, based on rather obscure evidence concluded actually occurred, has now become common knowledge within sectors of academia to the extent that it can be used in journal articles and books without any reference at all, neither to Bender, Hamblin, Larsson, nor others (see, for example, Adesman, 2009: 39; Coughlin and Zitarelli, 1984: 116; De Beuckelaer, 2002: 194; Gustavii, 2003: 116).
Honor where honor is due
Even though the decimal point error appears to have become common knowledge in some environments, there is no doubt that Hamblin in many cases gets his deserved credit for his ‘discovery’. There are, however, good reasons to question to what extent Hamblin appreciated statements such as this one:
Hamblin debunked the belief that spinach is a rich source of iron by tracing the Popeye-spinach myth to a mistake by the original investigators in the 1930’s who put the decimal point in the wrong place and made a ten-fold overestimate of iron content. (Skrabanek and McCormick, 1989: 32)
The quote is from a widely cited book, somewhat ironically entitled: Follies and Fallacies in Medicine. We have seen, however, that Hamblin was not at all the one who debunked the exaggerated values for iron in spinach, that the exaggeration took place long before the 1930s, and that there is no evidence indicating that the decimal point error ever occurred. If we go back to the first introductory quote from Larsson’s article, we will get an excellent example of this puzzling but not infrequent phenomenon in scientific publications: authors who independently of each other manage to commit the same set of errors. Larsson (1995) does not refer to Skrabanek and McCormick (1989) but makes exactly the same combination of mistakes. In addition, his wordings are strikingly similar to Skrabanek and McCormick’s. The great question is, of course, whether Larsson has actually consulted the source he directly referred to (i.e. Hamblin, 1981), or whether he took the message and the reference from a book that does not appear at all in his list of references: Skrabanek and McCormick (1989).
Skrabanek and McCormick are most likely the originators for the rather impressive concentration of errors that I have so far have attributed to Larsson. Whatever types or combination of shortcuts Larsson has taken, he has provided us with an excellent example of the topic contained in the title of his own article: ‘The dissemination of false data through inadequate citation’.
When academics plagiarize from each other, whether it is an idea or a reference, a single (and highly erroneous, as in this case) interpretation will appear as two or more mutually independent statements, reinforcing the reliability or truth value of each other in a way that is entirely undeserved. This is likely an important part of the explanation behind the fact that some academics no longer find it necessary to refer to a source when telling the story about the decimal point error. In the past 30 years, a large number of apparently independent sources have mutually confirmed the ‘fact’ that a decimal point error was made, whether it was in 1870, in the 1890s, or in the 1930s. Nothing indicates that the decimal point error ever was made, but the account about it will most likely live a long and colorful life, just like its parent myth, the belief that spinach is a good source of iron.
Patterns of irony
There seems to be a remarkably consistent pattern of irony in the ways various authors have dealt with the decimal point error. By using the non-existent, or at least undocumented, decimal point error as an illustration of the importance of accuracy and critical investigation, Bender, Hamblin, Larsson, and Skrabanek and McCormick all end up doing the opposite of what they are preaching.
Follies and Fallacies in Medicine by Skrabanek and McCormick (1989) deserves special attention, not just because it is probably the most widely distributed and read printed version of the decimal point error story. These authors are concerned with
distortions which set obstacles in the path of rational thought and enquiry. The progress of science and the growth of knowledge depend upon clearing away rubbish and challenging accepted dogma and belief. (Skrabanek and McCormick, 1989: 1)
When the third edition of the book was published in 1998, the original section on spinach was kept intact, including the rather obvious misrepresentations of what Hamblin actually wrote on the issue and his role as a ‘debunker’ of the exaggerated reports on iron content in spinach. James McCormick brought the irony to yet another level in the preface to the third edition:
One of the necessary tasks which preceded this preparation was to reread the previous (second) edition. The most surprising thing was that there did not appear to be anything which required deletion or modification. The Follies and Fallacies which we described are even more widespread than before. All I have been able to do is to add a small number of more recent references which demonstrate the persistence of sloppy thinking and foolish conclusion. It would be gratifying if we continue to be read. (Skrabanek and McCormick, 1998: ix)
Since the first edition was published in 1989, the book has been reprinted several times and translated to seven languages. According to Philip Steer, an Emeritus Professor who reviewed Follies and Fallacies as a ‘medical classic’ in British Medical Journal in 2008, the book ‘is on the reading list of medical schools around the world, to encourage an appropriate scepticism about medical dogma’ (Steer, 2008: 673).
Bender, Hamblin, Larsson, and Skrabanek and McCormick are all involved in projects trying to establish a boundary between science and knowledge, on the one hand, and bad science, non-science, or plain nonsense, on the other. In their endeavor, however, they end up violating a large number of basic academic principles for handling of knowledge, sometimes in outright spectacular ways. This puzzling pattern of irony may perhaps be illuminated by an observation made by Sergio Sismondo (2005), drawing upon Gieryn’s concept of ‘boundary-work’, in his article on the ‘Science Wars’:
When people discuss the work of competing fields, misrepresentations on a scale that would never be acceptable within their own field are perfectly ordinary. … philosophers are willing to take negative portrayals of competing fields at face value, and are uninterested in interrogating them closely. (p. 245)
Popeye always wins
In February 2011, an article titled ‘Popeye had it right: Spinach really does make you stronger’ was published in the web version of The Independent (2011). Here, a new nutrient, inorganic nitrate, pops into the scene, complicating the discussion on whether iron or vitamin A is the source of Popeye’s strength. This is a topic that is far beyond my own fields of competence, but I nonetheless found something interesting among the readers’ comments at the end of the article:
The story that the iron content of spinach was a myth based on a misplaced decimal point is itself a myth. Spinach has a lot of iron, just like other green vegetables, but it is unavailable for absorption.I should know, I was the one who was responsible for propagating the myth in a BMJ article.
The comment was signed ‘Terry Hamblin’. It will most likely do little to curb the spread of the academic urban legend that was born at Christmas time in 1981. Nonetheless, if the signature is authentic, there is something heroic about this public confession. This comment led me, for the first time, to click the ‘like’ button often found next to such readers’ comments.
The invisible heroes
There is something sad about all the urban legends that are brutally falsified or lose their edge because so many people find meaning in investigating them and publish their findings in books and on web pages for everyone to see. Urban legends are not just fascinating, entertaining, and colorful stories; they are also a part of our social communicative repertoire. When we lose them, we have to find other things to talk about, which are most likely not as funny, engaging, or bridge building as urban legends.
Nevertheless, I do not feel bad about having contributed to destroying the stories about spinach being a good source of iron, or the myth that a decimal point error made millions eat more spinach. Academic publications should have different standards and requirements regarding truth and accuracy from other mediums, and should not be a playground for rumors and urban legends. Accurate, complete, and relevant references to reliable sources are the best tools in order to avoid such a scenario.
This article may have drawn a rather dismaying picture of academic citation practices. There is, however, much to be admired in the ways various actors have approached the rumor that Hamblin accidentally turned into a piece of fact. Kronick sensed that the documentation was not as solid as we would expect, and included his small reservation. Sutton did a tremendous job, digging back to the 19th century and exposing for us all that Hamblin’s debunking of the myth about the exaggerated iron content was itself a myth. Another hero in this scenario is Hamblin himself. He admitted his mistake, not only on Sutton’s web page and in the commentary field of a spinach-related newspaper article. In December 2010, he published a detailed account titled ‘Spinach – I was right for the wrong reason’ on his own web site:
Now some fascinating research by Mike Sutton has found out the whole truth behind the decimal point and the iron in spinach myth and I am pleased to be able to say that I was right about spinach being useless as a source of iron, but utterly wrong about why the myth has taken hold. … The moral of this story is that a good story is not necessarily a true story. (Hamblin, 2010)
Hamblin committed a blunder more than 30 years ago, but he was doing his best until he died in February 2012 to stop the urban legend he was instrumental in creating. It was a hopeless fight, and the main reason is that many of us have a tendency to assume that everything we see in print is true, with or without a reference, as long as it is printed in a fairly respectable scientific journal.
Perhaps the greatest heroes are, however, invisible in this landscape: those who read about the decimal point error in one or more of these sources, but found out that they could not use it because the references and the documentation were not solid enough. Individuals with such attitudes are among the most important propellers of scientific development and accumulative knowledge, but many of them nonetheless end up as losers in systems where quantity is more important than quality, and where academic production is reduced to units being counted, rather than something worth taken into account.
An avalanche of low-quality research?
Are we experiencing an ‘avalanche of low-quality research’, or are the complaints about the current state of affairs primarily based on grumpy old-timers using the ‘good old days’ to kick the young and upcoming? Recently S. Rajasekaran (2012) expressed his worries in an article titled ‘Publish to flourish: Is it corrupting science?’ He laments that ‘the published paper is no longer sacred’ (p. 6), pointing to increasing incidences of fraud, plagiarism, and consequent retractions. Young researchers should be dissuaded ‘from joining the treadmill of the numbers game and encourage[d to pursue] focused publishing of high-quality research instead’ (p. 7). This is an argument in line with a large number of other commentators, but what is particularly welcome in Rajasekaran’s contribution is that he presents hard facts that are well suited to illustrate that there is good reason to ask the question of whether we are experiencing an ‘explosion of knowledge or just junk science’:
data show that only 45% of the articles published in the 4500 top scientific journals are cited within the first five years of publication, a figure that appears to be dropping steadily. … it appears that most published papers are inconsequential to science and simply pad the curricula vitae of researchers. (p. 5)
These are useful figures for those of us who may be accused of merely using anecdotal evidence when lamenting the current deterioration of academic standards. There is a problem, however, with this shockingly low figure and the enormous amount of time, energy, and money invested and apparently wasted in the remaining 55 percent articles that have not been cited 5 years after publication. The reference accompanying these figures leads us to Bauerlein et al. (2010) and their commentary we started out with in The Chronicle of Higher Education, ‘We must stop the avalanche of low-quality research’. This is obviously a secondary source, but even more worrisome, it is a paper that does not have any references at all. There is, however, enough information in the text to make it possible, with some creative searching, to make a fairly good guess at where the authors got the 45 percent figure from: a news article in Science (Hamilton, 1990), based on calculations from papers published between 1981 and 1985. In other words, Rajasekaran is using somewhat old figures to illustrate the miserable state of affairs in 2012.
The main problem with the 45 percent figure is not that we have to work hard in order to find out where it comes from, nor that it is old and rather irrelevant for the situation in 2012. We have in fact stumbled across yet another academic urban legend. The calculations behind the figure were based on a very heterogeneous composition of publications, including editorials, meeting abstracts, obituaries, notes, and letters, that is, types of works that are rarely cited, and for very good reasons (Pendlebury, 1991). A few years later, Eugene Garfield (1998), the founding father of the Institute for Scientific Information (ISI), wrote ‘I can only groan when I see errors perpetuated year after year’, pointing to the fact that Hamilton’s ‘misguided reports on uncitedness have unduly influenced many scholars and policy makers ever since’. Lariviere et al. (2009) state the claim ‘that most articles are never cited [is] a common lore that comes back periodically in the literature’ (p. 858), and their conclusions about uncited publications are in fact directly opposite to what Rajasekaran wrote 3 years later. Lariviere et al.’s data show that most scientific articles are in fact cited, and that rates of ‘citedness’ are not dropping at all, but are steadily increasing.
If we are concerned about the consequences of ‘low-quality research’, ‘junk science’, or the prevalence of academic urban legends, there are good reasons to shift the focus away from uncited and ignored publications, and worry instead about those that are cited, but which should not have been published. I will refrain from joining the debate on whether academic standards are deteriorating, or whether scientists are busier and take more shortcuts than before. What is clear, however, is that since 1981, the year Hamblin published his article about the decimal point error, the digitization of knowledge has dramatically improved the tools we have for enhancing the quality of research, including verification of references and documentation. The digital revolution has provided us with marvelous weapons for exposing and cutting down the prevalence rates of rumors and urban legends in academia. The problem is that the same devices can also be used for other purposes, such as contributing to what Haralambos Gavras (2002) has called the ‘Lazy author syndrome’: throwing a few keywords into a database to come up with an impressive list of references which at first glance cannot easily be exposed as secondary, irrelevant, unreliable, or sources not even read by the author.
The digital revolution has certainly made it easier to expose and debunk myths, but it has also created opportunities for new and remarkably efficient academic shortcuts, highly attractive and tempting not just in milieus characterized by increasing publication pressure and more concerned with quantity than quality, but also for groups and individuals strongly involved in rhetorics of demarcation of science, but less concerned with following the scientific principles they claim to defend. Some academic urban legends may perish in the new digital academic environment, but others will thrive and have ideal conditions for explosive growth.
Acknowledgments
I am grateful for valuable feedback and encouraging comments from Audun Ådland, Odd Mørkve, Kristian Heggenhougen, Brian Wood, and the anonymous reviewers of Social Studies of Science. I am particularly thankful to Sergio Sismondo for his detailed and constructive input, ranging from helping me out in my struggles with the English language to excellent suggestions of ways to organize the argument. I thank the editor of UNIPED, Yngve Nordkvelle, for encouraging me to develop an early Norwegian version of the paper (Rekdal, 2012) for an international audience.
Funding
The author received financial support from Bergen University College for the research and publication of this article. The write-up of this article was facilitated by a grant from Norsk faglitterær forfatterforening (NFFF) and a study loan from DNB, providing partial funding for a one-year stay (2012-2013) as Visiting Scholar at Department of Anthropology, Yale University.
Footnotes
1. Larsson’s article is written in a style common in medical journals, and the reference to Hamblin’s work appears as a numbered note in the text that I have here changed to a citation matching this journal’s style.
2. The reference will look different in other styles of referencing, but the principle of communicating clearly to the reader that the message comes from a secondary source remains the same. In medical journals following the AMA Manual of Style: A Guide for Authors and Editors (American Medical Association, 2007: 61), this particular reference will appear as a numbered note in the text, and the corresponding entry in the list of references will appear like this:
Hamblin, T. Fake! Br Med J. 1981;283(6307):1671-1674. Cited by: Larsson, K. The dissemination of false data through inadequate citation. J Intern Med. 1995;238(5):448-449.
3. For a collection of examples of the whisper game effect in academia, see Rekdal (in press-a, in press-b).
4. The myths of the decimal point error and of iron content in spinach carry a number of parallels to Malcolm Ashmore’s (1993) fascinating analysis of various diverging accounts of how Robert W. Wood allegedly debunked the existence of ‘N-rays’ in 1904.
References
Adesman A (2009) Baby Facts: The Truth about Your Child’s Health, from Newborn through Preschool. Hoboken, NJ: John Wiley & Sons.
American Medical Association (2007) AMA Manual of Style: A Guide for Authors and Editors, 10th edn. New York: Oxford University Press.
Ashmore M (1993) The theatre of the blind: Starring a Promethean prankster, a phoney phenomenon, a prism, a pocket, and a piece of wood. Social Studies of Science 23(1): 67–106.
Bauerlein M, Gad-el-Hak M, Grody W, McKelvey B, Trimble SW (2010) We must stop the avalanche of low-quality research. The Chronicle of Higher Education, 13 June. Available at: http://chronicle.com/article/We-Must-Stop-the-Avalanche-of/65890/ (accessed 20 March 2014).
Bender AE (1972) The Wider Knowledge of Nutrition. Inaugural Lecture by Arnold E. Bender. 24 October 1972, Queen Elizabeth College, University of London. London: Castle Cary Press.
Bender AE (1975a) The Facts of Food. London: Oxford University Press.
Bender AE (1975b) The meaning of nutrition in a changing society. Journal of the Canadian Dietetic Association 36(1): 137–149.
Bender AE (1977) Iron in spinach. The Spectator, 9 July, p. 18.
Bender AE, Bender DA (1982) Nutrition for Medical Students. Chichester: John Wiley & Sons.
Carroll AE, Vreeman RC (2009) Don’t Swallow Your Gum!: Myths, Half-Truths, and Outright Lies about Your Body and Health. New York: St. Martin’s Griffin.
Coughlin R, Zitarelli D (1984) The Ascent of Mathematics. New York: McGraw-Hill Book Company.
De Beuckelaer GM (2002) It’s Broken, Let’s Fix It: The Zeitgeist and Modern Enterprise. Berlin: Springer.
Frangoulis C, Carlotti F, Eisenhauer L, Zervoudaki S (2010) Converting copepod vital rates into units appropriate for biogeochemical models. Progress in Oceanography 84(1–2): 43–51.
Garfield E (1998) I had a dream … about uncitedness. The Scientist 12(14, 6 June): 10. Available at: http://www.the-scientist.com/?articles.view/articleNo/18998/title/I-Had-a-Dream—about-Uncitedness/ (accessed 20 March 2014).
Garrison C (ed.) (2009) The Iron Disorders Institute Guide to Anemia, 2nd edn. Nashville, TN: Cumberland House.
Gavras H (2002) Inappropriate attribution: The ‘Lazy Author Syndrome’. American Journal of Hypertension 15(9): 831.
Gustavii B (2003) How to Write and Illustrate a Scientific Paper. Cambridge: Cambridge University Press.
Hamblin TJ (1981) Fake! British Medical Journal 283(6307): 1671–1674. Available at: http://www.bmj.com/highwire/filestream/351132/field_highwire_article_pdf/0/1671 (accessed 20 March 2014).
Hamblin TJ (2010) Spinach – I was right for the wrong reason. In: Mutations of mortality: Random thoughts of Terry Hamblin about leukaemia, literature, poetry, politics, religion, cricket and music. Available at: http://mutated-unmuated.blogspot.com/2010/12/spinach-i-was-right-for-wrong-reason.html (accessed 19 March 2014).
Hamilton DP (1990) Publishing by – and for? – the numbers. Science 250(4986): 1331–1332.
Harzing A-W (2002) Are our referencing errors undermining our scholarship and credibility? The case of expatriate failure rates. Journal of Organizational Behavior 23(1): 127–148.
The Independent (2011) Popeye had it right: Spinach really does make you stronger. The Independent, 2 February. Available at: http://www.independent.co.uk/life-style/health-and-families/health-news/popeye-had-it-right-spinach-really-does-make-you-stronger-2201458.html (accessed 16 April 2013).
Kronick DA (1985) The Literature of the Life Sciences: Reading, Writing, Research. Philadelphia, PA: ISI Press.
Lariviere V, Gingras Y, Archambault E (2009) The decline in the concentration of citations, 1900–2007. Journal of the American Society for Information Science and Technology 60(4): 858–862.
Larsson KS (1995) The dissemination of false data through inadequate citation. Journal of Internal Medicine 238(5): 445–450.
Morrisey LJ (2004) Bibliometric and bibliographic analysis in an era of electronic scholarly communication. Science & Technology Libraries 22(3–4): 149–160.
Pendlebury DA (1991) Science, citation, and funding. Science 251(5000): 1410–1411.
Rajasekaran S (2012) Publish to flourish: Is it corrupting science? Bone & Joint360 1(4): 5–7.
Reader’s Digest Association (1988) Facts and Fallacies: Stories of the Strange and Unusual. Pleasantville, NY: Reader’s Digest Association.
Rekdal OB (2012) En vandring på akademiske snarveier. UNIPED 35(1): 44–59.
Rekdal OB (in press-a) Academic citation practice: A sinking sheep? Portal, Libraries and the Academy.
Rekdal OB (in press-b) Monuments to academic carelessness: The self-fulfilling prophecy of Katherine Frost Bruner. Science, Technology & Human Values.
Sismondo S (2005) Boundary work and the science wars: James Robert Brown’s Who Rules in science? Episteme 1(3): 235–248.
Skrabanek P, McCormick J (1989) Follies and Fallacies in Medicine. Glasgow: Tarragon Press.
Skrabanek P, McCormick J (1998) Follies and Fallacies in Medicine, 3rd edn. Whithorn: Tarragon Press.
Steer P (2008) Follies and fallacies in medicine. British Medical Journal 336(7645): 673.
Sutton M (2010a) Spinach, iron and Popeye: Ironic lessons from biochemistry and history on the importance of healthy eating, healthy scepticism and adequate citation. The Internet Journal of Criminology. Available at: http://www.internetjournalofcriminology.com/Sutton_Spinach_Iron_and_Popeye_March_2010.pdf (accessed 20 March 2014).
Sutton M (2010b) The spinach, Popeye, iron, decimal error myth is finally busted. BestThinking. Available at: http://www.bestthinking.com/articles/science/chemistry/biochemistry/the-spinach-popeye-iron-decimal-error-myth-is-finally-busted (accessed 20 March 2014).
Wetterer JK (2006) Quotation error, citation copying, and ant extinctions in Madeira. Scientometrics 67(3): 351–372.
Wright M, Armstrong JS (2008) The ombudsman: Verification of citations: Fawlty towers of knowledge? Interfaces 38(2): 125–139.
Biographies
Ole Bjørn Rekdal is Associate Professor at Bergen University College (Norway). He is a social anthropologist and has co-coordinated several health-related interdisciplinary research and competence-building programs in East Africa. His teaching and research activities are primarily within medical anthropology, and he has a special interest in citation practices, handling of knowledge, and information-seeking behavior in academia.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: June 12, 2014
Issue published: August 2014
Keywords
Rights and permissions
© The Author(s) 2014.
This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 3.0 License (http://www.creativecommons.org/licenses/by-nc/3.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (http://www.uk.sagepub.com/aboutus/openaccess.htm).
Request permissions for this article.
PubMed: 25272616
Authors
Metrics and citations
Metrics
Article usage*
Total views and downloads: 96268
*Article usage tracking started in December 2016
Altmetric
See the impact this article is making through the number of times it’s been read, and the Altmetric Score.
Learn more about the Altmetric Scores
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 42 view articles Opens in new tab
Crossref: 44
-
A simulation-based analysis of the impact of rhetorical citations in s...
-
Teaching and Learning a Joyful Citation Praxis: Affective Relations fo...
-
Turning good intentions into good outcomes: ethical dilemmas at a stud...
-
Until when did beavers historically inhabit the Iberian Peninsula? The...
-
Burden of proof: combating inaccurate citation in biomedical literatur...
-
Pygmies, Bushmen, and savage numbers – a case study in a sequence of b...
-
Citation Errors in Scientific Research and Publications: Causes, Conse...
-
On cell loss in Parkinson’s disease, and the citations that followed
-
Beyond facts – a survey and conceptualisation of claims in online disc...
-
Automated citation recommendation tools encourage questionable citatio...
-
Questionable Research Practices and Misconduct Among Norwegian Researc...
-
Do they practice what they preach? The presence of problematic citatio...
-
References
-
Knowing when to act: A call for an open misinformation library to guid...
-
Individual and collective identification in contemporary forensics
-
The ghost in the data: Evidence gaps and the problem of fake drugs in ...
-
Seven Myths on Crowding and Peripheral Vision
-
Peer Review or Lottery? A Critical Analysis of Two Different Forms of ...
-
Beyond one bone: Interdisciplinarity to assess nativeness of the tench...
-
A false image of health: how fake news and pseudo-facts spread in the ...
-
Symbolic Violence in Academic Life: A Study on How Junior Scholars are...
-
Scientific integrity issues in Environmental Toxicology and Chemistry:...
-
Quantifying flows and economies of informal e‐waste hubs: Learning fro...
-
Cuckoos: The holy grail of avian biodiversity conservation?
-
Literature practices: processes leading up to a citation
-
When “Normal” Becomes Normative: A Case Study of Researchers’ Quotatio...
-
Making the Case for Art and Culture: Persistent Challenges
-
Are Papers with Open Data More Credible? An Analysis of Open Data Avai...
-
Research Errors
-
Investigating the Role of Communication for Information Seekers’ Trust...
-
Sugar, spice and everything nice: how to end ‘sex testing’ in internat...
-
Amplifying the impact of open access: Wikipedia and the diffusion of s...
-
The social life of measurement: how methods have shaped the idea of cu...
-
Mantras of wildland fire behaviour modelling: facts or fallacies?
-
Dried fruit and dental health
-
Von Neumann was not a Quantum Bayesian
-
New straw for the old broom
-
Visa refusal following compulsory hospital admission under the ...
-
WHY (SOME) KNOWLEDGE IS THE PROPERTY OF A COMMUNITY AND POSSIBLY NONE ...
-
Northern Range Limit of Opuntia fragilis and the Cactaceae is 56°N, No...
-
Achieving human and machine accessibility of cited data in scholarly p...
-
Current Bibliography of the History of Science and Its Cultural Influe...
-
Monuments to Academic Carelessness...
-
Reconstructing the Black-Litterman Model
Figures and tables
Figures & Media
Tables
View Options
View options
PDF/ePub
View PDF/ePubGet access
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
