The Hawthorne effect in direct observation research with physicians and patients
Meredith A. Goodwin PhD
Research Associate
Department of Family Medicine and Community Health, Case Western Reserve University, Cleveland, Ohio, USA
Search for more papers by this authorKurt C. Stange MD, PhD
Professor
Department of Family Medicine and Community Health, Department of Epidemiology and Biostatistics, Department of Sociology, and Case Comprehensive Cancer Center, Case Western Reserve University, Cleveland, Ohio, USA
Search for more papers by this authorStephen J. Zyzanski PhD
Professor
Department of Family Medicine and Community Health, Department of Epidemiology and Biostatistics, and Case Comprehensive Cancer Center, Case Western Reserve University, Cleveland, Ohio, USA
Search for more papers by this authorBenjamin F. Crabtree PhD
Professor
Department of Family Medicine, Rutgers University, New Brunswick, New Jersey, USA
Search for more papers by this authorElaine A. Borawski PhD
Professor
Prevention Research Center for Healthy Neighborhoods and Department of Epidemiology and Biostatistics, and Case Comprehensive Cancer Center, Case Western Reserve University, Cleveland, Ohio, USA
Search for more papers by this authorCorresponding Author
Susan A. Flocke PhD
Professor
Department of Family Medicine and Community Health, Department of Epidemiology and Biostatistics, Prevention Research Center for Healthy Neighborhoods and Case Comprehensive Cancer Center, Case Western Reserve University, Cleveland, Ohio, USA
Correspondence
Susan A. Flocke, PhD, Case Western Reserve University, 11000 Cedar Ave., Suite 402, Cleveland, OH 44106, USA.
Email: [email protected]
Search for more papers by this authorMeredith A. Goodwin PhD
Research Associate
Department of Family Medicine and Community Health, Case Western Reserve University, Cleveland, Ohio, USA
Search for more papers by this authorKurt C. Stange MD, PhD
Professor
Department of Family Medicine and Community Health, Department of Epidemiology and Biostatistics, Department of Sociology, and Case Comprehensive Cancer Center, Case Western Reserve University, Cleveland, Ohio, USA
Search for more papers by this authorStephen J. Zyzanski PhD
Professor
Department of Family Medicine and Community Health, Department of Epidemiology and Biostatistics, and Case Comprehensive Cancer Center, Case Western Reserve University, Cleveland, Ohio, USA
Search for more papers by this authorBenjamin F. Crabtree PhD
Professor
Department of Family Medicine, Rutgers University, New Brunswick, New Jersey, USA
Search for more papers by this authorElaine A. Borawski PhD
Professor
Prevention Research Center for Healthy Neighborhoods and Department of Epidemiology and Biostatistics, and Case Comprehensive Cancer Center, Case Western Reserve University, Cleveland, Ohio, USA
Search for more papers by this authorCorresponding Author
Susan A. Flocke PhD
Professor
Department of Family Medicine and Community Health, Department of Epidemiology and Biostatistics, Prevention Research Center for Healthy Neighborhoods and Case Comprehensive Cancer Center, Case Western Reserve University, Cleveland, Ohio, USA
Correspondence
Susan A. Flocke, PhD, Case Western Reserve University, 11000 Cedar Ave., Suite 402, Cleveland, OH 44106, USA.
Email: [email protected]
Search for more papers by this authorAbstract
Rationale, aims, and objectives
This study examines the degree to which a “Hawthorne effect” alters outpatient-visit content.
Methods
Trained research nurses directly observed 4454 visits to 138 family physicians. Multiple data sources were used to examine the Hawthorne effect including differences in medical record documentation for observed visits and the prior visit by the same patient, time use during visits on the first versus the second observation day of each physician, and report by the patient, physician, and observer of the effect of observation.
Results
Visits on the first versus the second observation day were longer by an average of 1 minute (P < .001); there were time-use differences for 4 of 20 behaviour categories evaluated. No effect of the observer on the interaction was reported by 74% of patients and 55% of physicians. Most of those that reported an affect indicated it was slight. Patients with non-White race, lower-educational level, and poorer health were more likely to report being affected by the observer.
Conclusions
In a study that was designed to minimize the Hawthorne effect, the presence of an observer had little effect on most patient-physician visits but appeared to at least slightly effect a subgroup of vulnerable patients.
REFERENCES
- 1McPhee SJ, Richard RJ, Solkowitz SN. Performance of cancer screening in a university general internal medicine practice: comparison with the 1980 American Cancer Society guidelines. J Gen Intern Med. 1986; 1: 275-281.
- 2Montano DE, Phillips WR, Montaño DE. Cancer screening by primary care physicians: a comparison of rates obtained from physician self-report, patient survey, and chart audit. Am J Public Health. 1995; 85(6): 795-800.
- 3Stange KC, Zyzanski SJ, Jaén CR, et al. Illuminating the “black box”: a description of 4454 patient visits to 138 family physicians. J Fam Pract. 1998; 46(5): 377-389.
- 4Stange KCC, Zyzanski SJJ, Smith TFF, et al. How valid are medical records and patient questionnaires for physician profiling and health services research? Med Care. 1998; 36(6): 851-867.
- 5Cooper GS, Yuan Z, Stange KC, Dennis LK, Amini SB, Rimm AA. Agreement of Medicare claims and tumor registry data for assessment of cancer-related treatment. Med Care. 2000; 38(4): 411-421.
- 6Roethlisberger FJ, Dickson WJ. Management and the Worker. Cambridge, MA: Harvard University Press; 1961.
- 7Gillespie R. Manufacturing Knowledge: A History of the Hawthorne Experiments. Cambridge, MA: Cambridge University Press; 1991.
- 8Parsons HM. What happened at Hawthorne? Science. 1974; 183(4128): 922-930.
- 9Chiesa M, Hobbs S. Making sense of social research: how useful is the Hawthorne effect? Eur J Soc Psychol. 2008; 38(1): 67-74.
- 10Coleman T. Using video-recorded consultations for research in primary care: advantages and limitations. Fam Pract. 2000; 17(5): 422-427.
- 11Verstappen WHJM, van der Weijden T, ter Riet G, Grimshaw J, Winkens R, Grol RPTM. Block design allowed for control of the Hawthorne effect in a randomized controlled trial of test ordering. J Clin Epidemiol. 2004; 57(11): 1119-1123.
- 12McCarney R, Warner J, Iliffe S, van Haselen R, Griffin M, Fisher P. The Hawthorne effect: a randomised, controlled trial. BMC Med Res Methodol. 2007; 7: 30.
- 13Fernald DH, Coombs L, DeAlleaume L, West D, Parnes B. An assessment of the Hawthorne effect in practice-based research. J Am Board Fam Med. 2012; 25(1): 83-86.
- 14Srigley JA, Furness CD, Baker GR, Gardam M. Quantification of the Hawthorne effect in hand hygiene compliance monitoring using an electronic monitoring system: a retrospective cohort study. BMJ Qual Saf. 2014; 23(12): 974-980.
- 15Adair JG. The Hawthorne effect. a reconsideration of the methodological artifact. J Appl Psychol. 1984; 69: 334-345.
- 16Jones SR. Was there a Hawthorne effect? Am J Sociol. 1992; 98(3): 451-468.
- 17Pringle M, Stewart-Evans C. Does awareness of being video recorded affect doctors' consultation behavior. Br J Gen Pract. 1990; 40(340): 455-458.
- 18Martin E, Martin PML. The reactions of patients to a video camera in the consulting room. J R Coll Gen Pract. 1984; 34(268): 607-610.
- 19McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol. 2014; 67(3): 267-277.
- 20Rohrbaugh M, Rogers JC. What did the doctor do? When physicians and patients disagree. Arch Fam Med. 1994; 3: 125-129.
- 21Gerbert B, Hargreaves WA. Measuring physician behavior. Med Care. 1986; 24(9): 838-847.
- 22Suarez L, Goldman DA, Weiss NS. Validity of Pap smear and mammogram self-reports in a low-income Hispanic population. Am J Prev Med. 1995; 11(2): 94-98.
- 23McKenna MT, Speers M, Mallin K, Warnecke R. Agreement between patient self-reports and medical records for Pap smear histories. Am J Prev Med. 1992; 8(5): 287-291.
- 24Sawyer JA, Earp JA, Fletcher RH, Daye FEF, Wynn TM. Accuracy of women's self-report of their last Pap smear. Am J Public Health. 1989; 79(8): 1036-1037.
- 25Silagy C, Muir J, Coulter A, Thorogood M, Yudkin P, Roe L. Lifestyle advice in general practice: rates recalled by patients. Br Med J. 1992; 305: 871-874.
- 26Johnson CS, Archer J, Campos-Outcalt D. Accuracy of Pap smear and mammogram self-reports in a southwestern native American tribe. Am J Prev Med. 1995; 11(6): 360-363.
- 27Harlow SD, Linet MS. Agreement between questionnaire data and medical records: the evidence for accuracy of recall. Am J Epidemiol. 1989; 129(2): 233-247.
- 28Etzi S, Lane DS, Grimson R. The use of mammography vans by low-income women: the accuracy of self-reports. Am J Public Health. 1994; 84(1): 107-109.
- 29Degnan D, Harris R, Ranney J, Quade D, Earp JA, Gonzalez J. Measuring the use of mammography: two methods compared. Am J Public Health. 1992; 82(10): 1386-1388.
- 30Zapka JG, Bigelow C, Hurley T, et al. Mammography use among sociodemographically diverse women: the accuracy of self-report. Am J Public Health. 1996; 86(7): 1016-1021.
- 31Zuckerman AE, Starfield B, Hochreiter C, Kovasznay B. Validating the content of pediatric outpatient medical records by means of tape-recording doctor-patient encounters. Pediatrics. 1975; 56(3): 407-411.
- 32Romm FJ, Putnam SM. The validity of the medical record. Med Care. 1981; 19(3): 310-315.
- 33Starfield B, Steinwachs D, Morris I, Bause G, Siebert S, Westin C. Presence of observers at patient-practitioner interactions: impact on coordination of care and methodologic implications. Am J Public Health. 1979; 69(10): 1021-1025.
- 34Block MR, Schaffner KF, Coulehan JL. Ethical problems of recording physician-patient interactions in family practice settings. J Fam Pract. 1985; 21(6): 467-472.
- 35 U.S. Preventive Services Task Force. Guide to clinical preventive services. Vol 263. Baltimore: Agency for Healthcare Research and Quality; 1996. Available from http://www.uspreventiveservicestaskforce.org.
- 36Stange KC, Flocke SA, Goodwin MA, Kelly RB, Zyzanski SJ. Direct observation of rates of preventive service delivery in community family practice. Prev Med (Baltim). 2000; 31: 167-176.
- 37Callahan EJ, Bertakis KD. Development and validation of the Davis observation code. Fam Med. 1991; 23(1): 19-24.
- 38Kirschner CG, Burkett RC, Coy JA. Physicians' current procedural terminology: CPT ’94. 1994.
- 39Williams RL, Flocke SA, Stange KC. Race and preventive service delivery among African Americans and whites. Med Care. 2001; 38: 1260-1267.
- 40Flocke SA, Stange KC, Zyzanski SJ. The association of attributes of primary care with preventive service delivery. Med Care. 1997; 36: AS21-AS30.
- 41Ware J, Nelson E, Sherbourne C, Stewart A. Preliminary tests of a 6-item general health survey: a patient application. In: ASJ Ware, ed. Measuring Functioning and Well-Being. Durham, NC: Duke University Press; 1992.
- 42Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. JSTOR. 1995; 57(1): 289-300.
- 43Peugh JL. A practical guide to multilevel modeling. J Sch Psychol. 2010; 48(1): 85-112.
- 44 American Academy of Family Physicians. Facts About Family Practice. Kansas City, MO: American Academy of Family Physicians; 1996.
- 45Hall JA, Roter DL, Milburn MA, Daltroy LH. Patients' health as a predictor of physician and patient behavior in medical visits. A synthesis of four studies. Med Care. 1996; 34(12): 1205-1218.
- 46Shavers VL, Lynch CF, Burmeister LF. Racial differences in factors that influence the willingness to participate in medical research studies. Ann Epidemiol. 2002; 12: 248-256.
- 47Doescher MP, Saver BG, Franks P, Fiscella K. Racial and ethnic disparities in perceptions of physician style and trust. Arch Fam Med. 2000; 9(10): 1156-1163.
- 48Corbie-Smith G, Thomas SB, Williams MV, Moody-Ayers S. Attitudes and beliefs of African Americans toward participation in medical research. J Gen Intern Med. 1999; 14(9): 537-546.
- 49Fiscella K, Franks P, Clancy CM. Skepticism toward medical care and health care utilization. Med Care. 1998; 36(2): 180-189.
- 50Quam K. Evaluation of patient comfort during a videotaped office visit. Eval Program Plann. 1990; 13(4): 395-398.
- 51Hargreaves PN, Peppiatt R. Is videotaping of consultations acceptable to patients attending a hospice day centre? Palliat Med. 2001; 15(1): 49-54.
- 52Fairchild AL, Bayer R. Uses and abuses of Tuskegee. Science. 1999; 284(5416): 919-921.
- 53Robson S, Wardle J. Who's watching whom? A study of the effects of observers on group discussions. J Mark Res Soc. 1988; 30(3): 333-359.
- 54Ram P, Grol R, Rethans JJ, Schouten B, van der Vleuten CPM, Kester A. Assessment of general practitioners by video observation of communicative and medical performance in daily practice: issues of validity, reliability and feasibility. Med Educ. 1999; 33(6): 447-454.
