Free association dynamics predictive of individual differences in negative affectivity
To answer our first research question (Q1 in
Fig. 1A, i.e., whether the affective dynamics of spontaneous thought assessed with the FAST are predictive of individual differences in negative affectivity), we used the Markov chain analysis to create input features for machine learning to build a predictive model of general negative affectivity based on the transitional dynamics on the content dimensions. Before the analysis, we ensured that the postscan survey ratings reflected participants’ in-scanner experience by showing that (i) the in-scanner heart rates were substantially modulated by the levels of postscan valence ratings and (ii) the valence ratings were consistent with the emotion ratings intermittently obtained during the fMRI scans (see figs. S1 and S2 for detail).
We first defined discrete states for the Markov chain analysis by dividing each content dimension into two or three discrete states, as shown in
Fig. 2A. We divided valence, time, and safety-threat, which ranged from −1 to 1, into three discrete states (−1 to −0.33, −0.33 to 0.33, and 0.33 to 1; for valence, the three discrete states were negative/neutral/positive; for time, past/present/future; and for safety-threat, threatening/neutral/safe). For the self-relevance and vividness dimensions that ranged from 0 to 1, we divided them into two discrete states (0 to 0.5 and 0.5 to 1, which corresponded to low and high for both dimensions, respectively). We then calculated the transition probability, defined as the probability of making transitions from one to another discrete state on each dimension. We also calculated the steady-state probability, defined as the probability of converging to one state when the transition processes were sufficiently repeated. In addition to these dynamic features from the Markov chain analysis, we used each affective dimension’s mean and variance as predictor variables for the subsequent predictive modeling. Many of these dynamic features were relatively stable over a 7-week interval (for their test-retest reliability, see table S1).
For the predictive modeling, we used least absolute shrinkage and selection operator (LASSO) regression to predict individual differences in general negative affectivity with leave-one-subject-out cross-validation (LOSO-CV). The number of predictor variables for the final model was determined on the basis of the cross-validation performance with the training data (n = 62; one participant was excluded because of excessively few responses). We then tested the final model on three testing datasets to evaluate two different generalizability types—seed words and response modality. First, we tested the model on retest data with an average 7-week interval, in which a different set of seed words were used on a subset of participants from the training dataset (n = 30). Second, we tested the model on an independent test data (n = 117) collected from a new set of participants using a web-based FAST experiment, which collected the response through typing instead of speaking and with a longer time limit for a response (for more detail of the web-based FAST, please see Materials and Methods). Third, we tested the model on retest web-based FAST data with an average 7-week interval, in which, again, a different set of seed words were used on a subset of participants from the web-based FAST dataset (n = 49). As the outcome variable, we used factor scores from a factor analysis of subscales from self-report questionnaires measuring multiple aspects of general negative affectivity (for the details of the factor analysis results and self-report questionnaires, please refer to tables S2 to S4).
As shown in
Fig. 2B, the final predictive model showed significant prediction performance across four datasets; for the training dataset (
n = 62) with LOSO-CV, the prediction-outcome correlation between the actual and predictive values was
r = 0.481,
P = 0.0005, two-tailed, one-sample
t test; for the retest data (
n = 30) with different seed words,
r = 0.628,
P = 0.0016 (with LOSO-CV,
r = 0.501,
P = 0.0105); for the web-based FAST experiment (
n = 117),
r = 0.492,
P < 0.0001; and for the web-based FAST retest (
n = 49) with different seed words,
r = 0.569,
P = 0.0001. After removing outliers identified with three SDs, the prediction-outcome correlation remained to be significant (
r = 0.416,
P < 0.0001,
n = 115 after removing two outliers from the web-based FAST;
r = 0.445,
P = 0.0015,
n = 48 after removing one outlier participant from the web-based FAST retest). Given that these three independent test datasets had slightly different experimental parameters, such as seed words and response modality, these results demonstrated the robustness of the task and the predictive model.
We then examined the standardized beta coefficients of the 12 behavioral dynamic features to determine which predictors contributed significantly to the final model of general negative affectivity (
Fig. 2C). The beta coefficients indicated that the participants who showed (i) a higher variance of the safety-threat and valence scores; (ii) a higher transition probability from threatening to neutral states, (iii) a higher mean score on the time, vividness, and self-relevance dimensions; (iv) a higher transition probability from positive to negative states; and (v) a higher steady-state probability for the negative state were likely to report a higher level of general negative affectivity. Conversely, participants who showed (i) a higher variance on the self-relevance score, (ii) a higher transition probability from the negative or neutral to positive states, and (iii) a higher mean score on the valence dimension (i.e., more positive) tended to report a lower level of general negative affectivity. An additional analysis comparing the relative contributions of Markov chain–based features versus non–Markov chain features suggested that the Markov chain–based features (i.e., state transition dynamics) explained a substantial amount of variance above and beyond the non–Markov chain features (table S5).
We also trained an additional predictive model with a subset of the content dimension—valence, self-relevance, and time, and thus, we called this reduced model a VST model—to see whether these three dimensions were enough to predict the level of general negative affectivity. We chose these three dimensions because valence and self-relevance were highly correlated with safety-threat and vividness, respectively (fig. S3A). The VST model also showed significant predictions across four datasets with r = 0.409 to 0.677, P = 0.0042 to P < 0.0001, and seven of eight selected features overlapped with the original full model features (fig. S4).
Brain activation patterns during the concept reflection phase
To answer our second research question (Q2-1 in
Fig. 1A; identifying the brain representations of the content dimensions of spontaneous thought), we first examined brain activation patterns while participants were reflecting on the self-generated concepts in the context of their conceptual associations. We decided to make the concept reflection phase our main target for the fMRI analyses before we conducted the experiment because of the possibility of high levels of head motion during the concept generation phase (due to speaking, but see fig. S5 for the comparisons of head motion between the two phases). In addition, the concept reflection phase, in which each trial was 15 s long, could generate the brain representations of self-generated concepts more effectively than the concept generation phase, in which each trial was 2.5 s long including speaking.
As shown in
Fig. 3A, brain regions spanning the hippocampus, amygdala, and parts of the somatomotor network and default mode network engaged to a greater degree during conceptual self-reflection (warm color) compared to fixation baseline. In contrast, the visual network was engaged to a greater degree during baseline (cool color) than during conceptual self-reflection and thus appeared “deactivated” during reflection (see fig. S6 for the large-scale network definitions used for identification purposes).
Further investigation into the temporal shape of these hemodynamic response patterns using a finite impulse response (FIR) model revealed that the visual cortex “deactivation” was driven by a transient increase in activity around 3 s after the stimulus onset, followed by a large decrease afterward (
Fig. 3B).
K-means clustering on the FIR signal across the brain showed that the visual cortex, some brain regions within the ventral attention network, and the thalamus formed a cluster. This cluster (purple in
Fig. 3B) showed a transient activity right after the stimulus onset, likely reflecting perceptually guided and attentional orienting processes. Two additional clusters (green and yellow in
Fig. 3B) emerging from the clustering analysis mainly consisted of default mode and limbic network, lateral prefrontal cortex, and hippocampus and amygdala regions. Both clusters showed a delayed peak of brain activity around 7 to 10 s after the stimulus onset. Another cluster (red in
Fig. 3B) that had a large overlap with the somatomotor network showed a negative peak around 5 s after the stimulus onset. Given that the concept reflection did not involve any actual sensorimotor experience, this deactivation seemed reasonable. However, as shown in
Fig. 3B, the brain activation level within this cluster showed a slow recovery and turned into positive activation toward the end of the trial. Further characterization of the clusters with meta-analysis database and cortical hierarchy suggested that our task strongly engaged brain regions linked to autobiographical memory, conceptual processes, emotion, and autonomic regulation, which largely overlapped with the transmodal end in the principal gradient of cortical hierarchy (figs. S7 to S9).
Multivariate pattern–based predictive models of self-relevance and valence
To further investigate our second research question (“can we identify and decode the brain representations of affective qualities of spontaneous thought?” in
Fig. 1A), we developed whole-brain multivariate pattern–based predictive models for the content dimension ratings. To prepare training data, we grouped trials into quartiles representing four levels of each content dimension scale (for each participant) and then averaged the brain and rating data, resulting in four brain maps and four averaged rating scores per person for each dimension. After concatenating all participants’ data (
n = 61), we trained principal components regression (PCR) models for each content dimension and estimated model performance using two types of cross-validation methods—LOSO-CV and random-split cross-validation (RS-CV) (
31,
32). The cross-validated prediction performance was significant for self-relevance [correlation between actual and predicted ratings: with LOSO-CV, mean
r = 0.304,
z = 4.400,
P < 0.0001, two-tailed, bootstrap tests, mean squared error (mse) = 0.155; with RS-CV,
r = 0.276, mse = 0.156;
Fig. 4A], while other dimensions showed relatively poor prediction performance, with LOSO-CV, mean
r = 0.185, mse = 0.399 for valence; mean
r = 0.166, mse = 0.319 for safety-threat; mean
r = −0.064, mse = 0.228 for time; and
r = −0.015, mse = 0.182 for vividness. With RS-CV, mean
r = 0.152, mse = 0.427 for valence; mean
r = 0.147, mse = 0.323 for safety-threat; mean
r = −0.012, mse = 0.223 for time; and
r = −0.002, mse = 0.183 for vividness.
Among the dimensions that showed poor prediction performance, the valence result was unexpected because previous studies have shown reasonable performance in predicting positive versus negative emotional valence. For example, Chang
et al. (
33) reported that a whole-brain pattern–based marker could predict negative emotion ratings induced by pictures with high prediction performance. Other studies also reported that regional brain activity patterns could classify the positive versus negative valence with significant classification accuracy (
34,
35). However, unlike the previous studies, which used exogenous stimuli to evoke emotions, such as pictures (
33), movies (
34), or tastants (
35), the current study used self-generated, endogenous stimuli, which could have a potential impact on the semantic representations of emotional valence in the brain. Thus, we hypothesized that if we trained a predictive model only with the data from trials with low self-relevance scores, then we might be able to achieve a significant prediction performance similar to the previous studies. To test this hypothesis, we separately trained two models of valence, one for the low self-relevance trials (self-relevance scores ≤ 0.5) and the other for the high self-relevance trials (self-relevance scores > 0.5). Other analysis procedures were the same as the previous.
As hypothesized, we found that the valence model trained only on the low self-relevance trials (named the “valence–low-self” model;
Fig. 4B) showed a better and significant prediction performance, mean
r = 0.307,
z = 3.808,
P < 0.0001, bootstrap test, mse = 0.362 with LOSO-CV and
r = 0.303, mse = 0.364 with RS-CV, than the valence model trained on the data with high self-relevance, mean
r = 0.031,
z = 0.403,
P = 0.6872, mse = 0.448 with LOSO-CV and
r = 0.060, mse = 0.426 with RS-CV. To further validate this valence–low-self model, we tested the model on an independent study dataset from Chang
et al. (
33). We chose this study dataset because it used exogenous emotional stimuli to induce emotions [i.e., the International Affective Picture System (IAPS) pictures] and was publicly available from NeuroVault (
https://identifiers.org/neurovault.collection:503). As shown in
Fig. 5A, when we applied the valence–low-self model on the beta images corresponding to five-point negative emotion ratings ranging from 1 (neutral) to 5 (strongly negative), our model showed significant predictions across two independent datasets. The first dataset was the training data in the original study (
n = 121, mean
r = 0.225,
P < 0.00001, two-tailed, bootstrap tests) (
33), and the second dataset was the testing data in the original study (
n = 61,
r = 0.256,
P = 0.0002). These results provided evidence for the generalizability of our valence–low-self model to emotions evoked with exogenous visual stimuli.
To further understand the neurobiological meaning and validity of the predictive models, we visualized the thresholded predictive maps of self-relevance (
Fig. 4A) and valence–low-self models (
Fig. 4B) based on bootstrap tests with 10,000 iterations and the false discovery rate (FDR)
q < 0.05, identifying brain voxels that made reliable contributions to the prediction. For the self-relevance model, multiple brain regions within the default mode and limbic networks appeared to be important, including the medial prefrontal cortex (MPFC), posterior cingulate cortex (PCC), temporoparietal junction (TPJ), temporal pole (TP), hippocampus, and nucleus accumbens (NAc), consistent with previous literature (
16,
36–
39). Similarly, the valence–low-self model also identified important predictors within the default mode and limbic networks, such as the dorsomedial prefrontal cortex (DMPFC) and ventromedial prefrontal cortex (VMPFC), orbitofrontal cortex (OFC), and hippocampus. However, the predictive weight patterns within these regions were quite different between the self-relevance and valence–low-self models. For example, as shown in the insets of
Fig. 4, which presented the unthresholded weights of the self-relevance and valence–low-self models within the MPFC, the self-relevance model showed a negative → positive → negative gradient from dorsal to ventral parts of the MPFC. In contrast, the valence–low-self model showed a negative → positive gradient from dorsal to ventral MPFC. The pattern similarity of the unthresholded predictive weights within the MPFC between the two models was low,
r = 0.064. In addition to the default mode and limbic network regions, many voxels within the somatomotor and ventral and dorsal attention networks were among the important features of the models, suggesting that the information about the levels of self-relevance and valence involves many brain regions distributed across multiple brain systems.
z-scoring the outcome variables (i.e., self-relevance and valence scores) yielded similar predictive maps and results (fig. S10), suggesting that the within-subject variance was the main driver of the results.
Note that we did not further examine the predictive models of the other three dimensions, i.e., vividness, safety-threat, and time, given that the principal component analysis (PCA) results suggested three main principal components in the content dimensions. As shown in fig. S3A, the valence and safety-threat dimensions were highly correlated, and the self-relevance and vividness were also highly correlated. Therefore, by modeling valence and self-relevance, we should be able to cover the first two principal components. Regarding the time dimension, its predictive model did not perform well, and thus, we did not further examine the model here. In fig. S3B, we presented the univariate general linear model results with the three principal components.
Idiosyncratic brain representations of emotional valence for high self-relevance trials
One of the intriguing observations in the previous section was the poor prediction performance of the valence model when it was trained on the high self-relevance trials. We hypothesized that valence information for the high self-relevance trials would be represented with more idiosyncratic brain activity patterns than for the low self-relevance trials. To test the hypothesis, we first tested whether an a priori multivariate pattern–based emotion marker provided a similar pattern of results. We used the picture-induced negative emotion signature (PINES) (
33), which has shown its sensitivity and specificity in predicting the level of negative emotions across multiple studies (
33,
40). As presented in
Fig. 5B, the results were consistent with our findings in the previous section—the PINES was able to predict the valence ratings only for the data from the low self-relevance trials, mean
r = 0.194,
P = 0.0171, two-tailed, bootstrap tests, but not for the high self-relevance trials,
r = 0.081,
P = 0.2813, although the difference was not significant,
z = 1.260,
P = 0.208, 95% confidence interval = −0.163 to 1.266, two-tailed, bootstrap test with Fisher
z-transformation. These results suggest that the data from low self-relevance trials had some shared pattern information for valence common across participants, which the PINES could capture. In contrast, the valence information from high self-relevance trials cannot be decoded with the population-level emotion marker.
We then used an idiographic predictive modeling approach to quantifying the between-subject variability of predictive weights for high versus low self-relevance data. We first split the trials into high and low self-relevance groups (using 0.5 as a cutoff score). We then trained two valence models per person—one for high self-relevance trials and the other for low self-relevance trials using PCR with 5-fold cross-validation. Although the prediction performances were not different between the high versus low self-relevance predictive models (
t60 = 1.886,
P = 0.0641, two-tailed, paired
t test; the middle panel of
Fig. 6A), the SDs of the predictive weights across participants were significantly higher in the high self-relevance models than the low self-relevance models (
t211362 = 867.59,
P < 0.0001; the right panel of
Fig. 6A). Moreover, a group-level model trained on the high self-relevance data from all participants showed significantly worse prediction performance than the idiographic model built individually with the same data (
t60 = 10.005,
P < 0.0001, two-tailed, paired
t test; the middle panel of
Fig. 6A). These results provide additional converging evidence that the brain representations of emotional valence are shared across people when the stimulus is less self-relevant, but they become idiosyncratic across people when the stimulus is highly self-relevant.
We then examined where in the brain showed similar or distinct patterns of predictive weights between the valence models for high versus low self-relevance using a searchlight-based pattern similarity analysis method. As shown in
Fig. 6B, the brain areas that showed low pattern similarity (in blue, Bayes factor in favor of null hypothesis BF
01 > 6) were larger and more widely distributed across the whole brain than the brain areas that showed high pattern similarity (in red, Bayes factor in favor of alternative hypothesis BF
10 > 6). Brain regions with low pattern similarity included cortical and subcortical regions within the default mode and limbic networks, such as the VMPFC, perigenual anterior cingulate cortex, PCC, TP, hippocampus, and amygdala, and regions within the somatomotor network, such as supplementary motor area, right insula, and thalamus. Brain regions that showed high pattern similarity included the subgenual anterior cingulate cortex, left dorsal posterior insula, and right dorsal lateral prefrontal cortex. To summarize, these findings supported our hypothesis that the valence representations in the brain become more diverse and idiosyncratic across individuals as the stimuli become more self-relevant.