Generalizability
The trees, the forest, and the low-hanging fruit
Abstract
Clinical and epidemiologic investigations are paying increasing attention to the critical constructs of “representativeness” of study samples and “generalizability” of study results. This is a laudable trend and yet, these key concepts are often misconstrued and conflated, masking the central issues of internal and external validity. The authors define these issues and demonstrate how they are related to one another and to generalizability. Providing examples, they identify threats to validity from different forms of bias and confounding. They also lay out relevant practical issues in study design, from sample selection to assessment of exposures, in both clinic-based and population-based settings.
Get full access to this article
View all available purchase options and get full access to this article.
REFERENCES
1.
Brody BA. Readings in the Philosophy of Science. Englewood Cliffs, NJ: Prentice-Hall; 1970.
2.
Rothman KJ, Greenland S, Lash TL. Modern Epidemiology, 3rd ed. Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2008.
3.
Kruskal W, Mosteller F. Representative sampling: 4: the history of the concept in statistics, 1895–1939. Int Stat Rev 1980; 48: 169– 195.
4.
Kruskal W, Mosteller F. Representative sampling: 3: current statistical literature. Int Stat Rev 1979; 47: 245– 265.
5.
Kruskal W, Mosteller F. Representative sampling: 2: scientific literature, excluding statistics. Int Stat Rev 1979; 47: 111– 127.
6.
Kruskal W, Mosteller F. Representative sampling: 1: non-scientific literature. Int Stat Rev 1979; 47: 13– 24.
7.
Koepsell TD, Weiss NS. Epidemiologic Methods: Studying the Occurrence of Illness. Oxford: Oxford University Press; 2003.
8.
Tsuang D, Kukull W, Sheppard L, et al. Impact of sample selection on APOE epsilon 4 allele frequency: a comparison of two Alzheimer's disease samples. J Am Geriatr Soc 1996; 44: 704– 707.
9.
Farrer LA, Cupples LA, Haines JL, et al. Effects of age, sex, and ethnicity on the association between apolipoprotein E genotype and Alzheimer disease: a meta-analysis: APOE and Alzheimer Disease Meta Analysis Consortium. JAMA 1997; 278: 1349– 1356.
10.
Hendrie HC, Ogunniyi A, Hall KS, et al. Incidence of dementia and Alzheimer disease in 2 communities: Yoruba residing in Ibadan, Nigeria, and African Americans residing in Indianapolis, Indiana. JAMA 2001; 285: 739– 747.
11.
Ogunniyi A, Baiyewu O, Gureje O, et al. Epidemiology of dementia in Nigeria: results from the Indianapolis-Ibadan study. Eur J Neurol 2000; 7: 485– 490.
12.
Kukull WA. The association between smoking and Alzheimer's disease: effects of study design and bias. Biol Psychiatry 2001; 49: 194– 199.
13.
Chang CC, Zhao Y, Lee CW, Ganguli M. Smoking, death, and Alzheimer disease: a case of competing risks. Alzheimer Dis Assoc Disord Epub 2011.
14.
Mortimer JA, van Duijn CM, Chandra V, et al. Head trauma as a risk factor for Alzheimer's disease: a collaborative re-analysis of case-control studies: EURODEM Risk Factors Research Group. Int J Epidemiol 1991; 20 (suppl 2): S28– S35.
15.
Chandra V, Kokmen E, Schoenberg BS, Beard CM. Head trauma with loss of consciousness as a risk factor for Alzheimer's disease. Neurology 1989; 39: 1576– 1578.
16.
Tanner CM, Kamel F, Ross GW, et al. Rotenone, paraquat, and Parkinson's disease. Environ Health Perspect 2011; 119: 866– 872.
17.
Seidler A, Hellenbrand W, Robra BP, et al. Possible environmental, occupational, and other etiologic factors for Parkinson's disease: a case-control study in Germany. Neurology 1996; 46: 1275– 1284.
18.
Goldman SM, Tanner CM, Oakes D, Bhudhikanok GS, Gupta A, Langston JW. Head injury and Parkinson's disease risk in twins. Ann Neurol 2006; 60: 65– 72.
19.
Macciocchi SN, Barth JT, Alves W, Rimel RW, Jane JA. Neuropsychological functioning and recovery after mild head injury in collegiate athletes. Neurosurgery 1996; 39: 510– 514.
20.
Snowdon DA. Healthy aging and dementia: findings from the Nun Study. Ann Intern Med 2003; 139: 450– 454.
21.
Guskiewicz KM, Marshall SW, Bailes J, et al. Association between recurrent concussion and late-life cognitive impairment in retired professional football players. Neurosurgery 2005; 57: 719– 726.
22.
Plassman BL, Havlik RJ, Steffens DC, et al. Documented head injury in early adulthood and risk of Alzheimer's disease and other dementias. Neurology 2000; 55: 1158– 1166.
23.
Doll R, Hill AB. The mortality of doctors in relation to their smoking habits: a preliminary report. BMJ 1954; 1: 1451– 1455.
Information & Authors
Information
Published In
Copyright
Copyright © 2012 by AAN Enterprises, Inc.
Publication History
Received: October 17, 2011
Accepted: January 23, 2012
Published online: June 4, 2012
Published in print: June 5, 2012
Disclosure
The authors report no disclosures relevant to the manuscript. Go to Neurology.org for full disclosures.
Authors
Author Contributions
Walter A. Kukull, PhD, made substantive contribution to the design and conceptualization and interpretations and was responsible for the initial draft and revising the manuscript. Mary Ganguli, MD, MPH, made substantive contribution to the design and conceptualization and interpretations and contributed to revising the manuscript.
Metrics & Citations
Metrics
Citations
Download Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Select your manager software from the list below and click Download.
Cited By
- Longitudinal Changes and Associations Between Quantitative Sensory Testing and Psychological Factors in Whiplash-Associated Disorders: A Systematic Review and Meta-Analyses-Based Data Synthesis, The Journal of Pain, 25, 1, (12-30), (2024).https://doi.org/10.1016/j.jpain.2023.07.021
- A Systematic Review of the Implementation and Effectiveness of ‘The Daily Mile’ on Markers of Children’s Health, International Journal of Environmental Research and Public Health, 20, 13, (6203), (2023).https://doi.org/10.3390/ijerph20136203
- Perceived Stress and Impact on Role Functioning in University Students with Migraine-Like Headaches during COVID-19, International Journal of Environmental Research and Public Health, 20, 8, (5499), (2023).https://doi.org/10.3390/ijerph20085499
- Social Isolation and Social Support Influence Health Service Utilisation and Survival after a Cardiovascular Disease Event: A Systematic Review, International Journal of Environmental Research and Public Health, 20, 6, (4853), (2023).https://doi.org/10.3390/ijerph20064853
- Health Implications of Virtual Architecture: An Interdisciplinary Exploration of the Transferability of Findings from Neuroarchitecture, International Journal of Environmental Research and Public Health, 20, 3, (2735), (2023).https://doi.org/10.3390/ijerph20032735
- Toward consistent reporting of sample characteristics in studies investigating the biological mechanisms of romantic love, Frontiers in Psychology, 14, (2023).https://doi.org/10.3389/fpsyg.2023.983419
- A Critical Review of Neuropsychological Actuarial Criteria for Mild Cognitive Impairment, Journal of Alzheimer's Disease, 91, 1, (169-182), (2023).https://doi.org/10.3233/JAD-220805
- Arthroscopic partial meniscectomy for the degenerative meniscus tear: a comparison of patients included in RCTs and prospective cohort studies, Acta Orthopaedica, 94, (570-576), (2023).https://doi.org/10.2340/17453674.2023.24576
- Stigma and Stereotyping of Veterans who May Benefit from a Psychiatric Service Dog: A Test of the Stereotype Content Model and Weiner’s Attribution-Affect-Action Model, Journal of Veterans Studies, 9, 2, (41-57), (2023).https://doi.org/10.21061/jvs.v9i2.411
- Stigma and Stereotyping of Veterans who May Benefit from a Psychiatric Service Dog: A Test of the Stereotype Content Model and Weiner’s Attribution-Affect-Action Model, Journal of Veterans Studies, 9, 1, (73-89), (2023).https://doi.org/10.21061/jvs.v9i1.411
- See more
Loading...
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Personal login Institutional LoginPurchase Options
Purchase this article to get full access to it.