The Impact of Education on Depression Assessment: Differential Item Functioning Analysis
Abstract
A person’s level of education can affect their access to health care, and their health outcomes. Increasing rates of depression are another looming public health concern. Therefore, vulnerability is compounded for individuals who have a lower level of education and depression. Assessment of depressive symptoms is integral to many domains of health care including primary care and mental health specialty care. This investigation examined the degree to which education influences the psychometric properties of self-report items that measure depressive symptoms. This study was a secondary data analysis derived from three large internet panel studies. Together, the studies included the Beck Depression Inventory–II, the Center for Epidemiologic Studies Depression Scale, the Patient Health Questionnaire, and the Patient Reported Outcomes Measurement Information System measures of depression. Using a differential item functioning (DIF) approach, we found evidence of DIF such that some items on each of the questionnaires were flagged for DIF with effect sizes ranging from McFadden’s Pseudo R2 = .005 to .022. For example, results included several double-barreled questions flagged for DIF. Overall, questionnaires assessing depression vary in level of complexity, which interacts with the respondent’s level of education. Measurement of depression should include consideration of possible educational disparities, to identify people who may struggle with a written questionnaire, or may be subject to subtle psychometric biases associated with education.
Get full access to this article
View all access and purchase options for this article.
References
Andrade L., Caraveo-Anduaga J. J., Berglund P., Bijl R. V., De Graaf R., Vollebergh W., Dragomirecka E., Kohn R., Keller M., Kessler R. C., Kawakami N., Kiliç C., Offord D., Ustun T. B., Wittchen H.-U. (2003). The epidemiology of major depressive episodes: Results from the International Consortium of Psychiatric Epidemiology (ICPE) surveys. International Journal of Methods in Psychiatric Research, 12(1), 3-21. https://doi.org/10.1002/mpr.138
Baker D. P., Leon J., Smith Greenaway E. G., Collins J., Movit M. (2011). The education effect on population health: A reassessment. Population and Development Review, 37(2), 307-332. https://doi.org/10.1111/j.1728-4457.2011.00412.x
Bauldry S. (2015). Variation in the protective effect of higher education against depression. Society and Mental Health, 5(2), 145-161. https://doi.org/10.1177/2156869314564399
Beck A. T., Steer R. A., Brown G. K. (1996). Manual for the Beck Depression Inventory-II. Psychological Corporation.
Bjelland I., Krokstad S., Mykletun A., Dahl A. A., Tell G. S., Tambs K. (2008). Does a higher educational level protect against anxiety and depression? The HUNT study. Social Science & Medicine, 66(6), 1334-1345. https://doi.org/10.1016/j.socscimed.2007.12.019
Breslau J., Miller E., Breslau N., Bohnert K., Lucia V., Schweitzer J. (2009). The impact of early behavior disturbances on academic achievement in high school. Pediatrics, 123(6), 1472-1476. https://doi.org/10.1542/peds.2008-1406
Choi S. W., Gibbons L. E., Crane P. K. (2011). Lordif: An R package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and Monte Carlo simulations. Journal of Statistical Software, 39(8), 1-30. https://doi.org/10.18637/jss.v039.i08
Choi S. W., Schalet B., Cook K. F., Cella D. (2014). Establishing a common metric for depressive symptoms: Linking the BDI-II, CES-D, and PHQ-9 to PROMIS depression. Psychological Assessment, 26(2), 513-527. https://doi.org/10.1037/a0035768
Coleman M., Liau T. L. (1975). A computer readability formula designed for machine scoring. Journal of Applied Psychology, 60(2), 283-284. https://doi.org/10.1037/h0076540
Crane P. K., Gibbons L. E., Jolley L., van Belle G., Selleri R., Dalmonte E., De Ronchi D. (2006). Differential item functioning related to education and age in the Italian version of the Mini-Mental State Examination. International Psychogeriatrics, 18(3), 505-515. https://doi.org/10.1017/S1041610205002978
Crane P. K., Gibbons L. E., Ocepek-Welikson K., Cook K., Cella D., Narasimhalu K., Hays R. D., Teresi J. A. (2007). A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression. Quality of Life Research, 16(Suppl. 1), 69-84. https://doi.org/10.1007/s11136-007-9185-5
Dillman D. A., Smyth J. D., Christian L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method. John Wiley.
Flesch R. (1948). A new readability yardstick. Journal of Applied Psychology, 32(3), 221-233. https://doi.org/10.1037/h0057532
Fletcher J. M. (2008). Adolescent depression: Diagnosis, treatment, and educational attainment. Health Economics, 17(11), 1215-1235. https://doi.org/10.1002/hec.1319
Fletcher J. M. (2010). Adolescent depression and educational attainment: Results using sibling fixed effects. Health Economics, 19(7), 855-871. https://doi.org/10.1002/hec.1526
Gehlbach H. (2015). Seven survey sins. Journal of Early Adolescence, 35(5-6), 883-897. https://doi.org/10.1177/0272431615578276
Gideon L. (2012). The art of question phrasing. In Gideon L. (Ed.), Handbook of survey methodology for the social sciences (pp. 91-107). Springer.
Goldman D., Smith J. P. (2011). The increasing value of education to health. Social Science & Medicine, 72(10), 1728-1737. https://doi.org/10.1016/j.socscimed.2011.02.047
Green J. A., Cavanaugh K. L. (2015). Understanding the influence of educational attainment on kidney health and opportunities for improved care. Advances in Chronic Kidney Disease, 22(1), 24-30. https://doi.org/10.1053/j.ackd.2014.07.004
Gunning R. (1952). The technique of clear writing. McGraw-Hill.
Jones R. N., Gallo J. J. (2002). Education and sex differences in the Mini-Mental State Examination: Effects of differential item functioning. Journals of Gerontology: Series B: Psychological Sciences and Social Sciences, 57(6), P548-P558. https://doi.org/10.1093/geronb/57.6.P548
Kessler R. C., Petukhova M., Sampson N. A., Zaslavsky A. M., Wittchen H.-U. (2012). Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. International Journal of Methods in Psychiatric Research, 21(3), 169-184. https://doi.org/10.1002/mpr.1359
Kroenke K., Spitzer R. L., Williams J. B. W. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606-613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x
Lorant V., Deliege D., Eaton W., Robert A., Philippot P., Ansseau M. (2003). Socioeconomic inequalities in depression: A meta-analysis. American Journal of Epidemiology, 157(2), 98-112. https://doi.org/10.1093/aje/kwf182
McFadden D. (1974). Conditional logit analysis of qualitative choice behavior. In Zarembka P. (Ed.), Frontiers in econometrics (pp. 105-142). Academic Press. https://eml.berkeley.edu/reprints/mcfadden/zarembka.pdf
McLaren M. E., Szymkowicz S. M., Kirton J. W., Dotson V. M. (2015). Impact of education on memory deficits in subclinical depression. Archives of Clinical Neuropsychology, 30(5), 387-393. https://doi.org/10.1093/arclin/acv038
McLaughlin G. H. (1969). SMOG grading: A new readability formula. Journal of Reading, 12(8), 639-646. https://ogg.osu.edu/media/documents/health_lit/WRRSMOG_Readability_Formula_G._Harry_McLaughlin__1969_.pdf
Mezuk B., Eaton W. W., Golden S. H., Ding Y. (2008). The influence of educational attainment on depression and risk of type 2 diabetes. American Journal of Public Health, 98(8), 1480-1485. https://doi.org/10.2105/AJPH.2007.126441
Murden R. A., McRae T. D., Kaner S., Bucknam M. E. (1991). Mini-Mental State Exam scores vary with education in Blacks and Whites. Journal of the American Geriatrics Society, 39(2), 149-155. https://doi.org/10.1111/j.1532-5415.1991.tb01617.x
Paasche-Orlow M. K., Wolf M. S. (2007). The causal pathways linking health literacy to health outcomes. American Journal of Health Behavior, 31(Suppl. 1), S19-S26. https://doi.org/10.5555/ajhb.2007.31.supp.S19
Pandit A. U., Tang J. W., Bailey S. C., Davis T. C., Bocchini M. V., Persell S. D., Federman A. D., Wolf M. S. (2009). Education, literacy, and health: Mediating effects on hypertension knowledge and control. Patient Education and Counseling, 75(3), 381-385. https://doi.org/10.1016/j.pec.2009.04.006
Radloff L. S. (1977). The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385-401. https://doi.org/10.1177/014662167700100306
Ramirez M., Teresi J. A., Holmes D., Gurland B., Lantigua R. (2006). Differential item functioning (DIF) and the Mini-Mental State Examination (MMSE): Overview, sample, and issues of translation. Medical Care, 44(11, Suppl 3), S95-S106. https://doi.org/10.1097/01.mlr.0000245181.96133.db
Reeve B. B., Fayers P. (2005). Applying item response theory modeling for evaluating questionnaire item and scale properties. In Fayers P., Hays R. D. (Eds.), Assessing quality of life in clinical trials: Methods of practice (pp. 55-73). Oxford University Press.
Rinker T. (2017). Readability: Tools to calculate readability scores (Version 0.1.0) [Computer software]. https://doi.org/10.5281/zenodo.31950
Samejima F. (2016). Graded response models. In Handbook of item response theory (Vol. 1, pp. 95-107). CRC Press.
Shealy R., Stout W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58(2), 159-194. https://doi.org/10.1007/bf02294572
Sinkowitz-Cochran R. L. (2013). Survey design: To ask or not to ask? That is the question . . . Clinical Infectious Diseases, 56(8), 1159-1164. https://doi.org/10.1093/cid/cit005
Smith E. A., Senter R. J. (1967). Automated readability index (Technical Report AMRLTR-66-220). Wright-Patterson Air Force Base.
Stark S., Chernyshenko O. S., Drasgow F. (2004). Examining the effects of differential item (functioning and differential) test functioning on selection decisions: When are statistically significant effects practically important? Journal of Applied Psychology, 89(3), 497-508. https://doi.org/10.1037/0021-9010.89.3.497
Taple B. J., Griffith J. W., Wolf M. S. (2019). Interview administration of PROMIS depression and anxiety short forms. Health Literacy Research and Practice, 3(3), e196-e204. https://doi.org/10.3928/24748307-20190626-01
Teresi J. A., Kleinman M., Ocepek-Welikson K. (2000). Modern psychometric methods for detection of differential item functioning: Application to cognitive assessment measures. Statistics in Medicine, 19(11-12), 1651-1683. https://doi.org/10.1002/(sici)1097-0258(20000615/30)19:11/12%3C1651::aid-sim453%3E3.0.co;2-h
Teresi J. A., Ocepek-Welikson K., Kleinman M., Eimicke J. P., Crane P. K., Jones R. N., Lai J.-S., Choi S. W., Hays R. D., Reeve B. B., Reise S. P., Pilkonis P. A., Cella D. (2009). Analysis of differential item functioning in the depression item bank from the Patient Reported Outcome Measurement Information System (PROMIS): An item response theory approach. Psychology Science Quarterly, 51(2), 148-180
Vos T., Allen C., Arora M., Barber R. M., Brown A., Carter A., Casey D. C., Charlson F. J., Chen B. S., Coggeshall M., Cornaby L., Dandona L., Dicker D. J., Dilegge T., Erskine H. E., Ferrari A. J., Fitzmaurice C., Flemimg T., Forouzanfar M. H., . . . Murray J. L. (2016). Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990-2015: A systematic analysis for the Global Burden of Disease Study 2015. Lancet, 388(10053), 1545-1602. https://doi.org/10.1016/S0140-6736(16)31678-6
Zimmerman M., Coryell W. (1994). Screening for major depressive disorder in the community: A comparison of measures. Psychological Assessment, 6(1), 71-74. https://doi.org/10.1037/1040-3590.6.1.71
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: November 20, 2020
Issue published: March 2022
Keywords
PubMed: 33218257
Authors
Metrics and citations
Metrics
Article usage*
Total views and downloads: 774
*Article usage tracking started in December 2016
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 12 view articles Opens in new tab
Crossref: 0
-
The Clinical Relevance of a Socioecological Conceptualization of Self-...
-
Relationship between subjective well-being and depressive disorders: N...
-
Performance of a Rasch-based method for group comparisons of longitudi...
Figures and tables
Figures & Media
Tables
View Options
Get access
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
