Psychological Distress Across Adulthood: Equating Scales in Three British Birth Cohorts

We used data from four different data sets: a calibration sample and the 1946, 1958, and 1970 British birth cohorts.

Our main outcome was psychological distress. In the calibration sample, we collected data on all measures of psychological distress that were ever used in at least one of the three aforementioned birth cohorts: GHQ-12 and GHQ-28 (Goldberg & Williams, 1988), Malaise-9 (Rutter et al., 1970), PSE (Wing et al., 1967), and PSF scale (Lindelow et al., 1997; see Table S1 in the Supplemental Material available online), and administered them to the calibration sample. Following a counterbalanced design approach (Chen et al., 2009), we randomized the order of questionnaire administration to prevent order effects.

We used the GHQ-12 and Malaise-9 as our index measures for both scale-equating methods. Our choice of index measures was based on their widespread use within the birth cohorts and beyond: the Malaise Inventory was widely used in United Kingdom (UK)-population-based cohorts, and the GHQ-12 is used across a wider range of population-based and clinical studies. In addition, the two measures have different response options (binary yes/no for Malaise and a 4-point Likert scale for the GHQ-12), which results in differing ranges of possible scores and distributions. Using two index measures has the advantage of being able to assess robustness of the modeling procedure (each serves as the other’s sensitivity analysis). We collected and harmonized data on age in years, sex, and highest level of education (none, General Certificate of Secondary Education [GCSE] or equivalent, advanced levels [A-levels] or equivalent, degree or higher) to inform our multiple-imputation models.

We first composed descriptive statistics of all mental-health measures collected in the calibration sample and estimated correlations between these measures. In addition to multiple imputation, we used two approaches within an equipercentile-linking framework: equipercentile linking and applying a calibrated cutoff. For our equipercentile-linking approach to equating scales, we cross-tabulated percentile rankings on the GHQ-12 or Malaise-9 with percentile rankings on remaining measures to determine equivalent scores (Furukawa et al., 2019; Kolen & Brennan, 2014). Within the equipercentile-linking framework, we first identified a threshold score on the remaining measures most closely corresponding to that of the Malaise-9 (≥ 4) and the GHQ-12 (≥ 12). We applied this calibrated score back to the existing measures in the birth cohorts at each sweep to estimate the prevalence of mental distress. Throughout the article, these results are described under “calibrated cutoff.” Second, using our equipercentile ranking, we converted scores on other measures in all sweeps of the birth cohorts to GHQ-12 or Malaise-9 and estimated means, variance, and above-threshold prevalence of mental distress. These results are described as “equipercentile linking.” For details on how we met the various assumptions associated with the equipercentile-linking framework (Kolen & Brennan, 2014), see the Supplemental Material.

Separately, we used a multiple-imputation approach to equating scales. Multiple imputation is a more robust extension of linear transformation-based scale-equating approaches (Lamprianou, 2007; Sellers et al., 2019) and can account for stochastic error and uncertainty around single imputation transformed estimates. We coded data on covariates (age group, sex, level of education) and all psychological-distress measures identically across all four data sets; psychological-distress measures were coded at the scale level per age group. We appended the calibration sample to each of the cohort samples separately for GHQ-12 and Malaise-9. Each data set thus consisted of two samples and (at least) two measures per age group. In at least one of these samples (the calibration sample), both measures were complete, and data from this were used to impute values into the cohort sample (see Table S4 in the Supplemental Material). We used multiple imputation by fully conditional specification using chained equations (Little & Rubin, 2002). We used the following generic algorithm: impute measure A on the basis of measures B, C, sex, level of education, by age group. Analyses were conducted after imputation, combining estimates across 50 imputed data sets using Rubin’s rule (White et al., 2011). Given the large percentage of missing data, this larger number was chosen to reduce variability from the imputation procedure and to improve robustness of the imputation procedure. We estimated means, standard deviations, and above-threshold prevalence of mental distress across adulthood in the cohorts.

We assert that all procedures that contribute to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. All procedures that involved human subjects were approved by the University College London Institute of Education Research Ethics Committee (REC1210). Informed consent was obtained from all participants.

Scores calibrated to the GHQ-12 are detailed in Table 1, and calibrated means and standard deviations across the life course in each of the three birth cohorts are detailed in Table 2 and Figure 1a. Psychological distress peaked in the 1946 birth cohort at age 63 (M = 6.60, SD = 6.87) and declined to 5.19 (SD = 6.66) at age 69. In the 1958 birth cohort, psychological distress peaked at age 42 (M = 7.26, SD = 6.23). In the 1970 birth cohort, it peaked at age 26 (M = 8.40, SD = 4.93). Across the life course and across cohorts, distributions tended to be positively skewed (see Fig. S1 in the Supplemental Material).

Calibrated threshold scores corresponding to the original threshold score of the GHQ-12 (12) were 2 for Malaise-9 and GHQ-28, 3 for Malaise-24 and PSE, and 11 for the PSF (Table 1). Distribution of above-threshold prevalence is detailed in Figure 2. For the calibrated-cutoff scores, prevalence followed an inverse U shape in the 1946 birth cohort, peaking at 35.9% at age 63 before declining to 27.3% at age 69. In the 1958 birth cohort, the shape was similar to the 1946 cohort, and the peak was observed at age 42 (39.5%). Prevalence of psychological distress was relatively stable across the 1970 birth cohort, peaking at age 26 at 46.5%. Using the equipercentile-linking method (in which total scores were calibrated before the cutoff was applied), we found that the prevalence of psychological distress peaked in the 1946 birth cohort at age 63 at 35.9% before declining to 27.3% at age 69. In the 1958 birth cohort, prevalence peaked at age 42 at 23.1%, and in the 1970 birth cohort, the peak at age 42 was 28.7%.

Means and standard deviations of psychological distress across the life course, as derived from multiple imputation, are detailed in Table 1 and Figure 1a. In the 1946 cohort, mean scores peaked at age 43 (M = 11.41, SD = 0.35) before declining to 9.66 (SD = 0.19) at age 69. In the 1958 and 1970 birth cohorts, means were approximately similar across the life course, varying from 11.13 (SD = 0.35) at age 33 to 11.44 (SD = 0.20) at age 50 in the 1958 birth cohort and from 10.74 (SD = 0.04) at age 30 to 12.57 (SD = 0.42) at age 26 in the 1970 birth cohort.

Corresponding above-threshold prevalence estimates of psychological distress are detailed in Figure 2. The peak in the 1946 birth cohort occurred at age 43 at 44.6% before declining to 36.8% at age 69. In the 1958 birth cohort, prevalence declined from a peak at age 23 (45.1%) until age 42 (38.5%) before increasing again at age 50 (44.0%). In the 1970 birth cohort, prevalence was highest at age 26 (52.9%) and lowest at age 30 (34.5%).

Although broad patterns in distribution and above-threshold prevalence over adulthood appear similar across both index measures and the two scale-equating methods used in this article, point estimates differ substantially. For instance, for the mean scores and standard deviations at age 36 in the 1946 birth cohort, a threefold variance was observed for the GHQ calibration (equipercentile-linking method: M = 2.91, SD = 4.25; imputation approach: M = 9.98, SD = 0.38). Prevalence scores for this sweep varied from 2.7% in the equipercentile-linking and calibrated-threshold approaches using the GHQ-12 to 35.2% using a multiple-imputation approach.

Regarding cross-cohort comparisons of the prevalence of psychological distress calibrated against the GHQ-12, all approaches suggest that prevalence is highest in the 1970 birth cohort, although there is considerable variation in point estimates. Calibration against the Malaise-9 using a multiple-imputation approach suggests that prevalence in the 1946 birth cohort is higher than in the other two cohorts, whereas using both calibrated-cutoff and equipercentile-linking approaches suggests that prevalence is comparable across the three cohorts.

Full results for the various methods of calibration against the Malaise-9 can be found in the Supplemental Results in the Supplemental Material. Figure 1 details the mean scores across the adulthood sweeps in all three cohorts for both the equipercentile-linking and multiple-imputation methods for both Malaise-9 (Fig. 1a) and GHQ-12 (Fig. 1b). There appear to be larger differences in means between both methods for the GHQ-12 compared with the Malaise-9, although this was not formally tested. Calibration against the GHQ-12 yielded higher prevalence than calibration against the Malaise-9 (Fig. 2; see Fig. S6 in the Supplemental Material).

Although broad patterns of psychological distress across the life course were similar for GHQ-12 and Malaise-9 (e.g., prevalence was highest in midlife across the three cohorts), the curve was much flatter for the Malaise-9 across all methods (see Fig. S6 in the Supplemental Material). Whereas using the GHQ-12 as an index measure seems to suggest slightly higher psychological distress in the younger cohort, using Malaise-9 suggests that psychological distress is lowest in the 1958 cohort.

Results from additional sensitivity analyses comparing both calibration methods with original prevalence estimates at age 42 in the 1958 cohort and age 30 in the 1970 cohort suggested more accurate estimates using the multiple-imputation method with both measures (see Table S10 in the Supplemental Material). When calibrating against the GHQ-12, we found the equipercentile-linking method underestimated mean scores and prevalence in both cohorts. The multiple-imputation method was close to original estimates in both cohorts (1958 original: M = 11.06, SD = 4.79; multiple imputation: M = 11.19, SD = 0.05; 1970 original: M = 10.67, SD = 4.52; multiple imputation: M = 10.74, SD = 0.04). The calibrated-cutoff methods yielded relatively similar prevalence to the original estimate in the 1958 cohort (original prevalence: 36.1%; calibrated cutoff: 38.5%) but overestimated prevalence in the 1970 cohort (original prevalence: 32.9%; calibrated cutoff: 46.5%).

The previously reported (Bell, 2014; Gondek et al., 2020; Spiers et al., 2012) inverse U shape across adulthood was observed in the 1946 birth cohort for calibration against the GHQ-12 and Malaise-9 for the calibrated-cutoff and equipercentile-linking methods, although for both index measures, the multiple-imputation method showed a gradual decline in prevalence of psychological distress across the life course. This pattern was less clearly observed in the 1958 and 1970 cohorts across both index measures and calibration methods used, likely because these cohorts are still in middle age and prevalence of psychological distress has started to decline only marginally.

As in previous research comparing the age of 42 years across the 1958 and 1970 cohorts (Gondek et al., 2020; Ploubidis et al., 2017), we found that according to most methods and measures, the 1970 cohort has a higher prevalence of psychological distress at most ages compared with the 1958 cohort. However, we can draw no clear conclusions about any trends when also we also included the 1946 cohort. In the current study, we see lower prevalence in the 1946 cohort when the GHQ-12 was used as an index measure but higher prevalence of psychological distress in this cohort when the Malaise-9 was used as an index measure. Previous studies in North America have found that some older and more recent cohorts have higher distress—Keyes et al. (2014) demonstrated a U shape in between-cohorts effects—and other UK-based studies have observed lower prevalence in more recent cohorts (Thomson & Katikireddi, 2018). Note that the birth cohorts included in the present study were born across just 24 years in mid-20th-century Britain, and hence we cannot extrapolate findings to recent cohorts in which higher distress is increasingly reported. Our sensitivity analysis that used cohort sweeps in which both GHQ-12 and Malaise-9 were administered could not provide insight into why clear conclusions about cross-cohort trends could not be drawn. Although, it seems to suggest that multiple imputation might be more reliable than equipercentile linking. However, even just focusing on the multiple-imputation findings, we see prevalence of psychological distress increasing in more recent cohorts using the GHQ-12 yet lowest distress in the 1958 cohort using the Malaise-9.

These results should be interpreted in light of the strengths and limitations inherent to this study. We used three nationally representative birth cohorts, and our calibration sample had sufficient coverage across the whole distribution of mental health and was broadly representative of the current general population of the United Kingdom in terms of age, sex, level of education, and country of residence. Our study was methodologically robust and was designed to allow for assessing reliability across methods: We used two different index measures and scale-equating methods, which enabled us to describe differences on the basis of these. This is in contrast to previous literature that applied these methods and used only one method and measure (Choi et al., 2014; Fischer et al., 2011, 2012; Sellers et al., 2019), which resulted in limited evaluation of the reliability of any findings based on these approaches. Using an IRT calibration model as a further methodology was unsuitable because its key assumptions could not be met (Hambleton et al., 1991): The two questionnaires used as index measures do not have a common set of anchor items, and the remaining items do not have identical response options.

However, there are also some limitations inherent to our study design. Because we used data only from the United Kingdom, we are uncertain about the generalizability of our results to an international context. Mode of questionnaire administration differed between our calibration sample (all self-reported online) and the cohort samples (either self-report via a paper questionnaire or interviewer administered), and this might have led to higher reporting of mental-health symptoms in the calibration sample (Epstein et al., 2001). Finally, we used responses today to equate responses given at a previous point in time, as far back as the early 1980s. However, there appears to be no evidence that within-individuals and cross-cohort interpretation of the Malaise-9 changes over time (Ploubidis et al., 2019). In addition, measurement-invariance analyses (see Table S3 in the Supplemental Material) indicate that younger and older respondents today answer the measures used in this study similarly, which increases confidence in the longitudinal comparisons made.

Although life-course patterns of psychological distress were similar across both index measures and scale-equating methods, point estimates were not. Comparing methodologies, we found that the imputation method yielded higher means and standard deviations than the equipercentile-linking method, and sensitivity analyses indicate the former might be the less biased approach in this scenario. Although means are not directly comparable across index measures because of different scale ranges, prevalence estimates were higher using the GHQ-12.

These differences have little bearing on the longitudinal symptom profile (we confirmed an inverted U shape over the life course). There are two hypothetical explanations for this inverted U shape: It might be artifactual because the instruments we used are poor at capturing important aspects of mental health in later life, or it might be a reflection of genuine better mental health in later life (either through a reduced perception of pressure through socioemotional selectivity or through eudaemonic processes) after a period of greater stress in midlife (which potentially reflects the multiple stressors faced by many of child care, career pressures, and caring for elderly parents; Willis et al., 2010). However, these differences do have substantial implications for cross-cohort comparisons. For instance, examining means derived through the equipercentile method calibrated against the Malaise-9 (Fig. 1), we found that means are higher in the 1958 and 1970 cohorts, and any midlife peak is earlier in these cohorts. However, when using the same index measure and applying our multiple-imputation-based approach, the mean is highest in the 1946 birth cohort, and there is no discernible midlife peak in the other two cohorts. This method and measure dependency leaves us unable to make strong conclusions about whether more recent generations experience poorer mental health.

Compared with most of the other measures that we calibrated, the Malaise-9 has less variance (given the range from 0 to 9 compared with, e.g., the GHQ-12’s range of 0–36). We speculate that some of the discrepancies in the results we observe between these different measures might be due to differences in their scales (e.g., a score between 8 and 10 on the GHQ-12 gives a score of 1 on the Malaise-9; see Table S7 in the Supplemental Material). If this is indeed an important part of the consideration, then equating scales between measures with similar ranges and variance is more likely to yield reliable estimates than measures with vastly different variances. This might also explain why the multiple-imputation approach appears to be more reliable than the other two approaches because it does not superimpose substantially larger or smaller variance. An important implication of our findings is the need for formal statistical simulation studies to investigate the conditions in which different calibration and/or harmonization methods return unbiased results, especially if, as we have shown, bias occurs when measures with large discrepancy with respect to their range and standard deviation are calibrated.

This methodological and measure-based heterogeneity in point estimates across the life course and between cohorts calls into question the robustness of articles that used only one index measure to estimate mental-health outcomes, and we would recommend, as possible, for researchers who use scale-equating approaches to use two index measures with similar range. On the basis of our analyses, it would seem that multiple-imputation-based scale-equating methods are better suited to scenarios such as this, in which an external calibration sample is used.

The potential inadequacy of these scale-equating approaches in translating findings from one measure to another has considerable implications regarding recent announcements to mandate the use of certain measures in all research that pertains to certain common mental-health symptoms (Farber et al., 2020). The arguments in support for this include being able to use these measures to act as a common language, via scale-equating approaches, to be able to draw comparable comparisons and conclusions across a range of studies in various settings, including population studies, trials, and routine clinical monitoring. The findings here highlight that these approaches are problematic even for robust population-level inferences, and individual-level inferences (in which a score of X on this measure means a likely score of Y on the other measure for each individual) are likely to be even more error prone (Piantadosi et al., 1988).